Multicriteria Neural Network Design in the Speech-based Emotion Recognition Problem

Christina Brester, Eugene Semenkin, Maxim Sidorov, Olga Semenkina


In this paper we introduce the two-criterion optimization model to design multilayer perceptrons taking into account two objectives, which are the classification accuracy and computational complexity. Using this technique, it is possible to simplify the structure of neural network classifiers and at the same time to keep high classification accuracy. The main benefits of the approach proposed are related to the automatic choice of activation functions, the possibility of generating the ensemble of classifiers, and the embedded feature selection procedure. The cooperative multi-objective genetic algorithm is used as an optimizer to determine the Pareto set approximation in the two-criterion problem. The effectiveness of this approach is investigated on the speech-based emotion recognition problem. According to the results obtained, the usage of the proposed technique might lead to the generation of classifiers comprised by fewer neurons in the input and hidden layers, in contrast to conventional models, and to an increase in the emotion recognition accuracy by up to a 4.25% relative improvement due to the application of the ensemble of classifiers.


  1. Boersma, P., 2002. Praat, a system for doing phonetics by computer. Glot international, 5(9/10), pp. 341-345.
  2. Brester, Ch., Sidorov, M., Semenkin, E., 2014. Speechbased emotion recognition: Application of collective decision making concepts. Proceedings of the 2nd International Conference on Computer Science and Artificial Intelligence (ICCSAI2014), pp. 216-220.
  3. Brester, Ch., Semenkin, E., 2015a. Cooperative multiobjective genetic algorithm with parallel implementation. ICSI-CCI 2015, Part I, LNCS 9140, pp. 471-478.
  4. Brester, Ch., Semenkin, E., Sidorov, M., Kovalev, I., Zelenkov, P., 2015b. Evolutionary feature selection for emotion recognition in multilingual speech analysis // Proceedings of IEEE Congress on Evolutionary Computation (CEC2015), pp. 2406-2411.
  5. Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., and Weiss, B., 2005. A database of german emotional speech. In Interspeech, pp. 1517-1520.
  6. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T., 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6 (2), pp. 182-197.
  7. Eyben, F., W├Âllmer, M., and Schuller, B., 2010. Opensmile: the Munich versatile and fast opensource audio feature extractor. In Proceedings of the international conference on Multimedia, pp. 1459- 1462. ACM.
  8. Goutte, C., Gaussier, E. 2005. A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. ECIR'05 Proceedings of the 27th European conference on Advances in Information Retrieval Research, pp. 345-359.
  9. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I. H. The WEKA Data Mining Software: An Update. SIGKDD Explorations, Volume 11, Issue 1.
  10. Haq, S., Jackson, P., 2010. Machine Audition: Principles, Algorithms and Systems, chapter Multimodal Emotion Recognition. IGI Global, Hershey PA, pp. 398-423.
  11. Mori, H., Satake, T., Nakamura, M., and Kasuya, H., 2011. Constructing a spoken dialogue corpus for studying paralinguistic information in expressive conversation and analyzing its statistical/acoustic characteristics, Speech Communication, 53.
  12. Picard, R.W., 1995. Affective computing. Tech. Rep. Perceptual Computing Section Technical Report No. 321, MIT Media Laboratory, 20 Ames St., Cambridge, MA 02139.
  13. Schmitt, A., Ultes, S., and Minker, W., 2012. A parameterized and annotated corpus of the cmu let's go bus information system. Proceedings of International Conference on Language Resources and Evaluation (LREC), Istanbul, Turkey, pp. 3369-3373.
  14. Wang, R., 2013. Preference-Inspired Co-evolutionary Algorithms. A thesis submitted in partial fulfillment for the degree of the Doctor of Philosophy, University of Sheffield.
  15. Whitley, D., Rana, S., and Heckendorn, R., 1997. Island model genetic algorithms and linearly separable problems. Proceedings of AISB Workshop on Evolutionary Computation, vol.1305 of LNCS, pp. 109-125.
  16. Zhang, Q., Zhou, A., Zhao, S., Suganthan, P. N., Liu, W., Tiwari, S., 2008. Multi-objective optimization test instances for the CEC 2009 special session and competition. University of Essex and Nanyang Technological University, Tech. Rep. CES-487, 2008.
  17. Zitzler, E., Laumanns, M., Bleuler, S., 2004. A Tutorial on Evolutionary Multiobjective Optimization. In: Gandibleux X., (eds.): Metaheuristics for Multiobjective Optimisation. Lecture Notes in Economics and Mathematical Systems, vol. 535, Springer.
  18. Zitzler, E., Laumanns, M., Thiele, L., 2002. SPEA2: Improving the Strength Pareto Evolutionary Algorithm for Multiobjective Optimization. Evolutionary Methods for Design Optimisation and Control with Application to Industrial Problems EUROGEN 2001 3242 (103), pp. 95-100.

Paper Citation

in Harvard Style

Brester C., Semenkin E., Sidorov M. and Semenkina O. (2015). Multicriteria Neural Network Design in the Speech-based Emotion Recognition Problem . In Proceedings of the 12th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO, ISBN 978-989-758-122-9, pages 621-628. DOI: 10.5220/0005571806210628

in Bibtex Style

author={Christina Brester and Eugene Semenkin and Maxim Sidorov and Olga Semenkina},
title={Multicriteria Neural Network Design in the Speech-based Emotion Recognition Problem},
booktitle={Proceedings of the 12th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,},

in EndNote Style

JO - Proceedings of the 12th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,
TI - Multicriteria Neural Network Design in the Speech-based Emotion Recognition Problem
SN - 978-989-758-122-9
AU - Brester C.
AU - Semenkin E.
AU - Sidorov M.
AU - Semenkina O.
PY - 2015
SP - 621
EP - 628
DO - 10.5220/0005571806210628