Authors:
Marco Stricker
1
;
Syed Saqib Bukhari
1
;
Damian Borth
1
and
Andreas Dengel
2
Affiliations:
1
German Research Center for Artificial Intelligence (DFKI), Germany
;
2
German Research Center for Artificial Intelligence (DFKI) and Technical University of Kaiserslautern, Germany
Keyword(s):
Saliency Detection, Human Gaze, Adjective Noun Pairs, Neural Networks.
Related
Ontology
Subjects/Areas/Topics:
Artificial Intelligence
;
Biomedical Engineering
;
Biomedical Signal Processing
;
Computational Intelligence
;
Health Engineering and Technology Applications
;
Human-Computer Interaction
;
Methodologies and Methods
;
Neural Networks
;
Neurocomputing
;
Neurotechnology, Electronics and Informatics
;
Pattern Recognition
;
Physiological Computing Systems
;
Sensor Networks
;
Signal Processing
;
Soft Computing
;
Theory and Methods
;
Vision and Perception
Abstract:
This paper investigates if it is possible to increase the accuracy of Convolutional Neural Networks trained
on Adjective Noun Concepts with the help of saliency models. Although image classification reaches high
accuracy rates, the same level of accuracy is not reached for Adjective Noun Pairs, due to multiple problems.
Several benefits can be gained through understanding Adjective Noun Pairs, like automatically tagging large
image databases and understanding the sentiment of these images. This knowledge can be used for e.g. a
better advertisement system. In order to improve such a sentiment classification system a previous work
focused on searching saliency methods that can reproduce the human gaze on Adjective Noun Pairs and found
out that “Graph-Based Visual Saliency” belonged to the best for this problem. Utilizing these results we
used the “Graph-Based Visual Saliency” method on a big dataset of Adjective Noun Pairs and incorporated
these saliency data in the training p
hase of the Convolutional Neural Network. We tried out three different
approaches to incorporate this information in three different cases of Adjective Noun Pair combinations. These
cases either share a common adjective or a common noun or are completely different. Our results showed only
slight improvements which were not significantly better besides for one technique in one case.
(More)