A Hierarchical Approach for Multilingual Speech Emotion Recognition

Marco Nicolini, Stavros Ntalampiras

2023

Abstract

This article approaches the Speech Emotion Recognition (SER) problem with the focus placed on multilingual settings. The proposed solution consists in a hierarchical scheme the first level of which identifies the speaker’s gender and the second level predicts the speaker’s emotional state. We elaborate with three classifiers of increased complexity, i.e. k-NN, transfer learning based on YAMNet and Bidirectional Long Short-Term Memory neural networks. Importantly, model learning, validation and testing consider the full range of the big-six emotions, while the dataset has been assembled using well-known SER datasets representing six different languages. The obtained results show differences in classifying all data against only female or male data with respect to all classifiers. Interestingly, a-priori genre recognition can boost the overall classification performance.

Paper Citation

in Harvard Style

Nicolini M. and Ntalampiras S. (2023). A Hierarchical Approach for Multilingual Speech Emotion Recognition. In Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-626-2, pages 679-685. DOI: 10.5220/0011714800003411

in Bibtex Style

@conference{icpram23,
author={Marco Nicolini and Stavros Ntalampiras},
title={A Hierarchical Approach for Multilingual Speech Emotion Recognition},
booktitle={Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2023},
pages={679-685},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011714800003411},
isbn={978-989-758-626-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - A Hierarchical Approach for Multilingual Speech Emotion Recognition
SN - 978-989-758-626-2
AU - Nicolini M.
AU - Ntalampiras S.
PY - 2023
SP - 679
EP - 685
DO - 10.5220/0011714800003411