Sentiment Analysis from Sound Spectrograms via Soft BoVW and Temporal Structure Modelling

George Pikramenos, George Pikramenos, Georgios Smyrnis, Ioannis Vernikos, Ioannis Vernikos, Thomas Konidaris, Evaggelos Spyrou, Evaggelos Spyrou, Stavros Perantonis

2020

Abstract

Monitoring and analysis of human sentiments is currently one of the hottest research topics in the field of human-computer interaction, having many applications. However, in order to become practical in daily life, sentiment recognition techniques should analyze data collected in an unobtrusive way. For this reason, analyzing audio signals of human speech (as opposed to say biometrics) is considered key to potential emotion recognition systems. In this work, we expand upon previous efforts to analyze speech signals using computer vision techniques on their spectrograms. In particular, we utilize ORB descriptors on keypoints distributed on a regular grid over the spectrogram to obtain an intermediate representation. Firstly, a technique similar to Bag-of-Visual-Words (BoVW) is used, where a visual vocabulary is created by clustering keypoint descriptors, but instead a soft candidacy score is used to construct the histogram descriptors of the signal. Furthermore, a technique which takes into account the temporal structure of the spectrograms is examined, allowing for effective model regularization. Both of these techniques are evaluated in several popular emotion recognition datasets, with results indicating an improvement over the simple BoVW method.

Paper Citation

in Harvard Style

Pikramenos G., Smyrnis G., Vernikos I., Konidaris T., Spyrou E. and Perantonis S. (2020). Sentiment Analysis from Sound Spectrograms via Soft BoVW and Temporal Structure Modelling. In Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-397-1, pages 361-369. DOI: 10.5220/0009174503610369

in Bibtex Style

@conference{icpram20,
author={George Pikramenos and Georgios Smyrnis and Ioannis Vernikos and Thomas Konidaris and Evaggelos Spyrou and Stavros Perantonis},
title={Sentiment Analysis from Sound Spectrograms via Soft BoVW and Temporal Structure Modelling},
booktitle={Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2020},
pages={361-369},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0009174503610369},
isbn={978-989-758-397-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Sentiment Analysis from Sound Spectrograms via Soft BoVW and Temporal Structure Modelling
SN - 978-989-758-397-1
AU - Pikramenos G.
AU - Smyrnis G.
AU - Vernikos I.
AU - Konidaris T.
AU - Spyrou E.
AU - Perantonis S.
PY - 2020
SP - 361
EP - 369
DO - 10.5220/0009174503610369