Saliency based Adjective Noun Pair Detection System

Marco Stricker, Syed Saqib Bukhari, Damian Borth, Andreas Dengel

2018

Abstract

This paper investigates if it is possible to increase the accuracy of Convolutional Neural Networks trained on Adjective Noun Concepts with the help of saliency models. Although image classification reaches high accuracy rates, the same level of accuracy is not reached for Adjective Noun Pairs, due to multiple problems. Several benefits can be gained through understanding Adjective Noun Pairs, like automatically tagging large image databases and understanding the sentiment of these images. This knowledge can be used for e.g. a better advertisement system. In order to improve such a sentiment classification system a previous work focused on searching saliency methods that can reproduce the human gaze on Adjective Noun Pairs and found out that “Graph-Based Visual Saliency” belonged to the best for this problem. Utilizing these results we used the “Graph-Based Visual Saliency” method on a big dataset of Adjective Noun Pairs and incorporated these saliency data in the training phase of the Convolutional Neural Network. We tried out three different approaches to incorporate this information in three different cases of Adjective Noun Pair combinations. These cases either share a common adjective or a common noun or are completely different. Our results showed only slight improvements which were not significantly better besides for one technique in one case.

Paper Citation

in Harvard Style

Stricker M., Bukhari S., Borth D. and Dengel A. (2018). Saliency based Adjective Noun Pair Detection System.In Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-275-2, pages 387-394. DOI: 10.5220/0006578303870394

in Bibtex Style

@conference{icaart18,
author={Marco Stricker and Syed Saqib Bukhari and Damian Borth and Andreas Dengel},
title={Saliency based Adjective Noun Pair Detection System},
booktitle={Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2018},
pages={387-394},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006578303870394},
isbn={978-989-758-275-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Saliency based Adjective Noun Pair Detection System
SN - 978-989-758-275-2
AU - Stricker M.
AU - Bukhari S.
AU - Borth D.
AU - Dengel A.
PY - 2018
SP - 387
EP - 394
DO - 10.5220/0006578303870394