Emotion Recognition based on Heart Rate and Skin Conductance

Mickael Ménard, Paul Richard, Hamza Hamdi, Bruno Daucé, Takehiko Yamaguchi

2015

Abstract

Information on a customer’s emotional states concerning a product or an advertisement is a very important aspect of marketing research. Most studies aimed at identifying emotions through speech or facial expressions. However, these two vary greatly with people’s talking habits, which cause the data lacking continuous availability. Furthermore, bio-signal data is also required in order to fully assess a user’s emotional state in some cases. We focused on recognising the six basic primary emotions proposed by Ekman using biofeedback sensors, which measure heart rate and skin conductance. Participants were shown a series of 12 video-based stimuli that have been validated by a subjective rating protocol. Experiment results showed that the collected signals allow us to identify user's emotional state with a good ratio. In addition, a partial correlation between objective and subjective data has been observed.

References

  1. Ang, K., Wahab, A., Quek, H. & Khosrowabadi, R., 2010. Eeg-based emotion recognition using self-organizing map for boundary detection. In: I. C. Society, ed. Proceedings of the 2010 20th International Conference on Pattern Recognition,. Washington, DC, USA: s.n., p. 4242-4245.
  2. Barreto, A. & Zhai, J., 2006. Stress detection in computer users through noninvasive monitoring of physiological signals. In: Biomedical Science Instrumentation. s.l.:s.n., p. 495-500.
  3. Bartolini, E. E., 2011. Eliciting Emotion with Film: Development of a Stimulus Set, Wesleyan University: s.n.
  4. Bednarski, J. D., March 29, 2012. Eliciting Seven Discrete Positive Emotions Using Film Stimuli, Vanderbilt University: s.n.
  5. Bercher, J.-F. & Baudoin, G., 2001. Transformée de Fourier Discrète,» École Supérieure d'Ingénieurs en Électrotechnique et Électronique. s.l.:s.n.
  6. Carvalho, S., Leite, J., Galdo-Álvarez, S. & Gonçalves, O., 2012. The Emotional Movie Database (EMDB): a self-report and psychophysiological study., Neuropsychophysiology Lab Cipsi, School of Psychology, University of Minho, Campus de Gualtar, Braga, Portugal.: s.n.
  7. Croucher, M. & Sloman, A., 1981. Why robots will have emotions. Originally appeared in Proceedings IJCAI 1981 ed. Vancouver, Sussex University as Cognitive Science: s.n.
  8. Darwin, C., 1872. The expression of emotion in man and animal. Chicago: University of Chicago Press (reprinted in 1965).
  9. Ekman, P., 1999. In: Basic emotions. New York: Sussex U.K.: John Wiley and Sons, Ltd, p. 301-320.
  10. Ekman, P. & Friesen, W., 1982. Measuring facial movement with the facial action coding system. In: Emotion in the human face (2nd ed.). s.l.:New York: Cambridge University Press.
  11. Friesen, W. & Ekman, P., 1978. Facial Action Coding System: A Technique for Measurement of Facial Movement.. Palo Alto, California: Consulting Psychologists Press.
  12. Gray, J., 1982. The neuropsychology of anxiety - an inquiry into the functions of the septo-hippocampal system. s.l.:s.n.
  13. Hamdi, H., 2012. Plate-forme multimodale pour la reconnaissance d'émotions via l'analyse de signaux physiologiques : Application à la simulation d'entretiens d'embauche, Université d'Angers, France: s.n.
  14. Hamdi, H., Richard, P., Suteau, A. & Saleh, M., 2011. Virtual reality and affective computing techniques for face-to-face communication. In: GRAPP 2011 - Proceedings of the International Conference on Computer Graphics Theory and Applications. Algarve, Portugal: s.n., p. 357-360.
  15. Healey, J., Vyzas, E. & Picard, R., 2001. Toward machine emotional intelligence: Analysis of affective physiological state. In: IEEE Transactions on Pattern Analysis and Machine Intelligence. s.l.:s.n., p. 1175- 1191.
  16. Herbelin, B. et al., 2004. Using physiological measures for emotional assessment: a computer-aided tool for cognitive and behavioural therapy. In: ICDVRAT. Oxford, England: s.n., p. 307-314.
  17. I-Maginer, n.d. Nonin medical wristox2. [Online]
  18. Izard, C. E., 1971. The face of emotion. New York: Appleton-Century-Crofts.
  19. James, W., 1884. What is an emotion?,. s.l.:s.n.
  20. Kim, J., 2007. Bimodal emotion recognition using speech and physiological changes, s.l.: s.n.
  21. Lang, P., 1995. The emotion probe: Studies of motivation and attention. In: American psychologist. s.l.:s.n., p. 372-385.
  22. Levenson, R., 1988. Emotion and the autonomic nervous system: a prospectus for research on autonomic specificity. Social Psychophysiology and Emotion: Theory and Clinical Applications, p. 17-42.
  23. Levenson, R. W., 2003. Autonomic specificity and emotion. s.l.:s.n.
  24. Lin, C.-J. & Chang, C.-C., 2011. LIBSVM : a library for support vector machines. ACM Transactions on Intelligent Systems and Technology.
  25. Mauss, I. & Robinson, M., 2009. Measures of emotion: A review. In: Cognition & Emotion. s.l.:s.n., p. 209-237.
  26. McDougall, W., 1926. An introduction to social psychology. s.l.:s.n.
  27. Moradi, M. & Khalili, Z., 2009. Emotion recognition system using brain and peripheral signals: using correlation dimension to improve the results og EEG. In: I. Press, ed. Proceedings of the 2009 international joint conference on Neural Networks. Piscataway, NJ, USA: s.n., p. 1571-1575.
  28. Mower, O., 1930. Learning theory and behavior. s.l.:s.n.
  29. Oatley, K. & Johnson-Laird, P., 1987. Towards a cognitive theory of emotions. s.l.:s.n.
  30. Ortony, A. & Turner, W., 1990. What's basic about basic emotions. In: Psychological Review. s.l.:s.n., p. 315- 331.
  31. Panksepp, J., 1982. Toward a general psycho-biological theory of emotions. In: Behavioral and Brain Sciences. s.l.:s.n., p. 407-422.
  32. Pantic, M. & Gunes, H., 2010. Automatic, dimensional and continuous emotion recognition. In: Int'l Journal of Synthetic Emotion. s.l.:s.n., p. 68-99.
  33. Pfeifer, R., Kaiser, S. & Wehrle, T., 1988. In Cognitive Perspectives on Emotion and Motivation. Artificial intelligence models of emotions, Volume 44, p. 287- 320.
  34. Picard, R., 1995. Affective Computing, rapport interne du MIT Media Lab, TR321, Massachusetts Institute of Technology, Cambridge, USA: s.n.
  35. Plutchik, R., 1980. A general psychoevolutionary theory of emotion. In: Emotion: Theory, research, and experience. s.l.:New York: Academic, pp. 3-33.
  36. Pun, T., Ansari-Asl, K. & Chanel, G., 2007. Valencearousal evaluation using physiological signals in an emotion recall paradigm. In: Systems, Man and Cybernetics. Montreal, Que, 2007: IEEE International Conference, p. 2662-2667.
  37. Rothkrantz, L. & Pantic, M., 2003. Toward an affectsensitive multimodal human-computer interaction. Proceedings of the IEEE, Volume 91, p. 1370-1390.
  38. Russell, A., 1979. Affective space is bipolar. In: Personality and Social Psychology. s.l.:s.n., p. 345- 356.
  39. Saleh, M., Suteau, A., Richard, P. & Hamdi, H., 2011. A multi-modal virtual environment to train for job interview. In: PECCS 2011 - Proceedings of the International Conference on Pervasive and Embedded Computing and Communication Systems. Algarve, Portugal: s.n., p. 551-556.
  40. Schaefer, A., Nils, F., Sanchez, X. & Philippot, P., 2004. A multi-criteria assessment of emotional films, University of Louvain, Louvain-La-Neuve, Belgium.: s.n.
  41. TEA, n.d. [Online], Available at: http://www.teaergo.com/ Tomkins, S., 1984. Chapter Affect Theory. In: Approaches to emotion. Erlbaum, Hillsdale, NJ: s.n., p. 163-195.
  42. Westerink, J. H. & van den Broek, E. L., 2009. Guidelines for affective signal processing (asp): from lab to life. In: I. C. Society, ed. In Proceedings of the Interaction and Workshops, ACII 2009. Amsterdam: s.n., p. 704- 709.
Download


Paper Citation


in Harvard Style

Ménard M., Richard P., Hamdi H., Daucé B. and Yamaguchi T. (2015). Emotion Recognition based on Heart Rate and Skin Conductance . In Proceedings of the 2nd International Conference on Physiological Computing Systems - Volume 1: PhyCS, ISBN 978-989-758-085-7, pages 26-32. DOI: 10.5220/0005241100260032


in Bibtex Style

@conference{phycs15,
author={Mickael Ménard and Paul Richard and Hamza Hamdi and Bruno Daucé and Takehiko Yamaguchi},
title={Emotion Recognition based on Heart Rate and Skin Conductance},
booktitle={Proceedings of the 2nd International Conference on Physiological Computing Systems - Volume 1: PhyCS,},
year={2015},
pages={26-32},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005241100260032},
isbn={978-989-758-085-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 2nd International Conference on Physiological Computing Systems - Volume 1: PhyCS,
TI - Emotion Recognition based on Heart Rate and Skin Conductance
SN - 978-989-758-085-7
AU - Ménard M.
AU - Richard P.
AU - Hamdi H.
AU - Daucé B.
AU - Yamaguchi T.
PY - 2015
SP - 26
EP - 32
DO - 10.5220/0005241100260032