Detecting Thermal Emotional Profile

Yang Fu, Claude Frasson

2016

Abstract

Human can react emotionally to specific situations provoking some physiological changes that can be detected using a variety of devices, facial expression, electrodermal activity, and EEG systems are among the efficient devices which can assess the emotional reactions. However, emotions can trigger some small changes in blood flow with an impact on skin temperature. In the present research we use EEG and a thermal camera to determine the emotional profile of a user submitted to a set of emotional pictures. Six experiments were performed to study the thermal reactions to emotions, and in each experiment, 80 selected standard stimuli pictures of 20 various emotional profiles from IAPS (a database of emotional images) were displayed to participants every three seconds. An infrared camera and EEG were used to capture both thermal pictures of participants and their electrical brain activities. We used several area of the face to train a classifier for emotion recognition using Machine Learning models. Results indicate that some specific areas are more significant than others to show a change in temperature. These changes are also slower than with the EEG signal. Two methods were used to train the HMM, one is training classifier per the participant self data (participant-independent), another is training classifier based on all participants` thermal data (participant-dependent). The result showed the later method brings more accuracy emotion recognition.

References

  1. Ahn, Hyung-il, Teeters, Alea, Wang, Andrew, Breazeal, Cynthia, & Picard, Rosalind. (2007). Stoop to Conquer: Posture and affect interact to influence computer users' persistence, The 2nd International Conference on Affective Computing and Intelligent Interaction, September 12-14, Lisbon, Portugal.
  2. Anderson, James. (2001). Tailoring Assessment to Study Student Learning Styles. In American Association for Higher Education, (53) 7.
  3. Baker, Lawrence M.; Taylor, William M. (1954). The relationship under stress between changes in skin temperature, electrical skin resistance, and pulse rate. Journal of Experimental Psychology, Vol 48(5), Nov 1954, 361-366.
  4. Balaras, C. A., &Argiriou, A. A. (2002). Infrared thermography for building diagnostics. Energy and buildings, 34(2), 171-183.
  5. Bos, Danny Oude, (2006). EEG-based Emotion Recognition, The Influence of Visual and Auditory Stimuli. Department of Computer Science, University of Twente.
  6. Chaffar, Soumaya, Derbali, Lotfi, & Frasson Claude. (2009). Towards Emotional Regulation in Intelligent Tutoring Systems, AACE World Conference on Elearning in Corporate, Government, Healthcare& Higher Education: E-LEARN 2009, Vancouver, Canada.
  7. Derbali, Lotfi& Frasson, Claude. (2012). Assessment of Learners' Motivation during Interactions with Serious Games: a Study of some Motivational Strategies in Food-Force. Advances in Human-Computer Interaction - Special issue on User Assessment in Serious Games and Technology-Enhanced Learning. Volume 2012, January 2012. Article No. 5.
  8. Devereaux, M. D., Parr, G. R., Thomas, D. P., &Hazleman, B. L. (1985). Disease activity indexes in rheumatoid arthritis; a prospective, comparative study with thermography. Annals of the Rheumatic Diseases, 44(7), 434-437.
  9. Doherty, Patrick &Rudol, Piotr. (2007) A UAV Search and Rescue Scenario with Human Body Detection and Geolocalization, Volume 4830 of the series Lecture Notes in Computer Science pp 1-13.
  10. Doucleff, Michaeleen. (2013). Mapping Emotions On The Body: Love Makes Us Warm All Over, Health News From NPR, December 30, 2013.
  11. Frasson, C., Brosseau, Pierre-Olivier, Thi Hong Dung Tran. Virtual Environment for Monitoring Emotional Behaviour in Driving. The 12th International Conference On Intelligent Tutoring Systems (ITS 2014). Honolulu, Hawaii. June 5-9, 2014.PDF.
  12. Heraz, Alicia, Razaki, Ryad. & Frasson, Claude. (2007) Using machine learning to predict learner emotional state from brainwaves. 7th IEEE conference on Advanced Learning Technologies: ICALT 2007, Niigata, Japan, 2007.
  13. Heraz, Alicia, Razaki, Ryad. & Frasson, Claude. (2009) How Do Emotional Stimuli Influence the Learner's Brain Activity? Tracking the brainwave frequency bands Amplitudes. International Conference on Agents and Artificial Intelligence. ICAART, Jan 2009. Porto, Portugal.
  14. Ioannou, Stephanos, Ebisch, Sjoerd, Aureli, Tiziana, Bafunno, Daniela, Ioannides, Helene Alexi, Cardone, Daniela, Manini, Barbara, Romani, Gian Luca, Gallese, Vittorio, &Merla, Arcangelo. (2013) The Autonomic Signature of Guilt in Children: A Thermal Infrared Imaging Study. Published: November 19, 2013. DOI: 10.1371/journal.pone.0079440.
  15. Jatupaiboon, Noppadon, Pan-ngum, Setha, &Israsena, Pasin. (2013). Real-Time EEG-Based Happiness Detection System, The Scientific World Journal, vol. 2013, Article ID 618649, 12 pages, 2013. doi:10.1155/2013/618649.
  16. Jones, B.F.(1998) A Reappraisal of the Use of Infrared Thermal Image Analysis in Medicine, IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 17, NO. 6, DECEMBER 1998, 1019-1027.
  17. Kapoor, Ashish, Ahn, Hyungil, & Picard, Rosalind W. (2005) Mixture of Gaussian Processes for Combining Multiple Modalities, in Proceedings of Multiple Classifier Systems, Eds. N. C. Oza, R. Polikar, J. Kittler, and F. Roli, 6th International Workshop, MCS 2005, June 2005, Seaside, CA, pp. 86-96.
  18. Kuraoka, Koji,Nakamura, Katsuki. (2011) The use of nasal skin temperature measurements in studying emotion in macaque monkeys. Physiology & Behavior Volume 102, Issues 3-4, 1 March 2011, Pages 347-355.
  19. Lang, Peter J. (2008). International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical Report A-8. University of Florida, Gainesville, FL. Bradley, M.M., & Cuthbert, B.N. (2008).
  20. Liu, Y., Sourina, O., & Nguyen, M. K. (2010, October). Real-time EEG-based human emotion recognition and visualization. 2010 International Conference on Cyberworlds (CW)(pp. 262-269). IEEE.
  21. Murugappan, M., Nagarajan, R., &Yaacob, S. (2011). Combining spatial filtering and wavelet transform for classifying human emotions using EEG Signals. Journal of Medical and Biological Engineering, 31(1), 45-51.
  22. Nkambou, R.V. (2004). Facial expression analysis for emotion recognition in ITS. In: ITS'2004 workshop on Emotional Intelligence proceedings.
  23. Ogorevc, J., Pušnik, I., Geršak, G., Bojkovski, J., &Drnovšek, J. (2015). Thermal imaging in medicine. ZdravniškiVestnik, 84(11), 757-770.
  24. Osgood, C.E. (1962). Studies on the generality of affective meaning systems. American Psychologist, 17, 10-28.
  25. Panayiotou, G. (2008). Emotional dimensions reflected in ratings of affective scripts. Personality and Individual Differences, 44(8), 1795-1806.
  26. Ring, E. F. J., &Ammer, K. (2012). Infrared thermal imaging in medicine. Physiological measurement, 33(3), R33.
  27. Rimm-Kaufman, S. E., & Kagan, J. (1996). The psychological significance of changes in skin temperature. Motivation and Emotion, 20(1), 63-78.
  28. Spalding, S. J., Kwoh, C. K., Boudreau, R., Enama, J., Lunich, J., Huber, D., .& Hirsch, R. (2008). Threedimensional and thermal surface imaging produces reliable measures of joint shape and temperature: a potential tool for quantifying arthritis. Arthritis Research and Therapy, 10(1), R10.
  29. Titman, D. J. (2001), Applications of thermography in nondestructive testing of structures, NDT & E International, 34(2), 149-154.
  30. Vos, Pieter, De Cock, Paul, Munde, Vera, Petry, Katja, Noortgate, Wim Van Den , Bea, &Maes B. (2012) The tell-tale: what do heart rate; skin temperature and skin conductance reveal about emotions of people with severe and profound intellectual disabilities? Res Dev Disabil. 2012 Jul-Aug; 33(4): 1117-27.
  31. Wilson, P R, Low, P A, Bedder, M D, Covigton, W E C, and Rauck, R. (1996) Diagnostic algorithm for complex regional pain syndromes Reflex Sympathetic Dystrophy ed A Re-appraisal, W Jänig and M StantonHicks (Seattle: IASP Press) pp 93-105.
Download


Paper Citation


in Harvard Style

Fu Y. and Frasson C. (2016). Detecting Thermal Emotional Profile . In Proceedings of the 3rd International Conference on Physiological Computing Systems - Volume 1: PhyCS, ISBN 978-989-758-197-7, pages 142-151. DOI: 10.5220/0006007901420151


in Bibtex Style

@conference{phycs16,
author={Yang Fu and Claude Frasson},
title={Detecting Thermal Emotional Profile},
booktitle={Proceedings of the 3rd International Conference on Physiological Computing Systems - Volume 1: PhyCS,},
year={2016},
pages={142-151},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006007901420151},
isbn={978-989-758-197-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Physiological Computing Systems - Volume 1: PhyCS,
TI - Detecting Thermal Emotional Profile
SN - 978-989-758-197-7
AU - Fu Y.
AU - Frasson C.
PY - 2016
SP - 142
EP - 151
DO - 10.5220/0006007901420151