Biometrics for Emotion Detection (BED): Exploring the combination of Speech and ECG

Marleen H. Schut, Kees Tuinenbreijer, Egon L. van den Broek, Joyce H. D. M. Westerink

2010

Abstract

The paradigm Biometrics for Emotion Detection (BED) is introduced, which enables unobtrusive emotion recognition, taking into account varying environments. It uses the electrocardiogram (ECG) and speech, as a powerful but rarely used combination to unravel people's emotions. BED was applied in two environments (i.e., office and home-like) in which 40 people watched 6 film scenes. It is shown that both heart rate variability (derived from the ECG) and, when people's gender is taken into account, the standard deviation of the fundamental frequency of speech indicate people's experienced emotions. As such, these measures validate each other. Moreover, it is found that people's environment can indeed of influence experienced emotions. These results indicate that BED might become an important paradigm for unobtrusive emotion detection.

References

  1. Cohn, J.F.: Foundations of human computing: Facial expression and emotion. Lecture notes in Artificial Intelligence (Artifical Intelligence for Human Computing) 4451 (2007) 1-16
  2. Kim, J., André, E.: Emotion recognition using physiological and speech signal in short-term observation. Lecture Notes in Computer Science (Perception and Interactive Technologies) 4021 (2006) 53-64
  3. Kim, J.: 15. In: Bimodal emotion recognition using speech and physiological changes. Vienna, Austria: I-Tech Education and Publishing (2007) 265-280
  4. Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Transactions on Pattern Analysis and Machine Intelligence 30 (2008) 2067-2083
  5. Kleinginna, P.R., Kleinginna, A.M.: A categorized list of emotion definitions, with a suggestion for a consensual definition. Motivation and Emotion 5 (1981) 345-379
  6. Russel, J.A., Barrett, L.F.: Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant. Journal of Personality and Social Psychology 26 (1999) 805-819
  7. van den Broek, E.L., Westerink, J.H.D.M.: Considerations for emotion-aware consumer products. Applied Ergonomics 40 (2009) 1055-1064
  8. van den Broek, E.L., LisÉ, V., Janssen, J.H., Westerink, J.H.D.M., Schut, M.H., Tuinenbreijer, K. In: Affective Man-Machine Interface: Unveiling human emotions through biosignals. Volume 52 of Communications in Computer and Information Science. Berlin/Heidelberg, Germany: Springer-Verlag (2010) 21-47
  9. Fairclough, S.H.: Fundamentals of physiological computing. Interacting with Computers 21 (2009) 133-145
  10. Ververidis, D., Kotropoulos, C.: Emotional speech recognition: Resources, features, and methods. Speech Communication 48 (2006) 1162-1181
  11. Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31 (2009) 39-58
  12. Gunes, H., Piccardi, M.: Automatic temporal segment detection and affect recognition from face and body display. IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics 39 (2009) 64-84
  13. Pantelopoulos, A., Bourbakis, N.G.: A survey on wearable sensor-based systems for health monitoring and prognosis. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 40 (2010) 1-12
  14. Berntson, G.G., Bigger, J.T., Eckberg, D.L., Grossman, P., Kaufmann, P.G., Malik, M., Nagaraja, H.N., Porges, S.W., Saul, J.P., Stone, P.H., van der Molen, M.W.: Heart rate variability: Origins, methods, and interpretive caveats. Psychophysiology 34 (1997) 623-648
  15. Appelhans, B.M., Luecken, L.J.: Heart rate variability as an index of regulated emotional responding. Review of General Psychology 10 (2006) 229-240
  16. Boersma, P.P.G., Weenink, D.J.M.: Praat 4.0.4 (2006) URL: http://www.praat.org [Last accessed on January 13, 2010].
  17. Barrett, L.F.: Valence as a basic building block of emotional life. Journal of Research in Personality 40 (2006) 35-55
Download


Paper Citation


in Harvard Style

H. Schut M., Tuinenbreijer K., L. van den Broek E. and H. D. M. Westerink J. (2010). Biometrics for Emotion Detection (BED): Exploring the combination of Speech and ECG . In Proceedings of the 1st International Workshop on Bio-inspired Human-Machine Interfaces and Healthcare Applications - Volume 1: B-Interface, (BIOSTEC 2010) ISBN 978-989-674-020-7, pages 59-66. DOI: 10.5220/0002813400590066


in Bibtex Style

@conference{b-interface10,
author={Marleen H. Schut and Kees Tuinenbreijer and Egon L. van den Broek and Joyce H. D. M. Westerink},
title={Biometrics for Emotion Detection (BED): Exploring the combination of Speech and ECG},
booktitle={Proceedings of the 1st International Workshop on Bio-inspired Human-Machine Interfaces and Healthcare Applications - Volume 1: B-Interface, (BIOSTEC 2010)},
year={2010},
pages={59-66},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002813400590066},
isbn={978-989-674-020-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 1st International Workshop on Bio-inspired Human-Machine Interfaces and Healthcare Applications - Volume 1: B-Interface, (BIOSTEC 2010)
TI - Biometrics for Emotion Detection (BED): Exploring the combination of Speech and ECG
SN - 978-989-674-020-7
AU - H. Schut M.
AU - Tuinenbreijer K.
AU - L. van den Broek E.
AU - H. D. M. Westerink J.
PY - 2010
SP - 59
EP - 66
DO - 10.5220/0002813400590066