The Smart Sensor Integration Framework and its Application in EU Projects

Johannes Wagner, Frank Jung, Jonghwa Kim, Thurid Vogt, Elisabeth André

Abstract

Affect sensing by machines is an essential part of next-generation human-computer interaction (HCI). However, despite the large effort carried out in this field during the last decades, only few applications exist, which are able to react to a user's emotion in real-time. This is certainly due to the fact that emotion recognition is a challenging part in itself. Another reason is that so far most effort has been put towards offline analysis and only few applications exist, which can react to a user's emotion in real-time. In response to this deficit we have developed a framework called Smart Sensor Integration (SSI), which considerably jump-starts the development of multimodal online emotion recognition (OER) systems. In this paper, we introduce the SSI framework and describe how it is successfully applied in different projects under grant of the European Union, namely the CALLAS and METABO project, and the IRIS network.

References

  1. Bailenson, J.; Pontikakis, E.; Mauss, I.; Gross, J.; Jabon, M.; Hutcherson, C.; Nass, C.; John, C.: Real-time classification of evoked emotions using facial feature tracking and physiological responses. Int'l Journal of Human-Computer Studies, 66(5):303-317, 2008.
  2. Caridakis, G.; Raouzaiou, A.; Karpouzis, K.; Kollias, S.: Synthesizing gesture expressivity based on real sequences. In Proc. of LREC Workshop on multimodal corpora: from multimodal behaviour theories to usable models, 2006.
  3. Charles, F.; Pizzi, D.; Cavazza, M.; Vogt, T.; André, E.: Emotional input for character-based interactive storytelling. In The 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), Budapest, Hungary, 2009.
  4. Gilroy, S. W.; Cavazza, M.; Chaignon, R.; Mäkelä, S.-M.; Niranen, M.; André, E.; Vogt, T.; Urbain, J.; Billinghurst, M.; Seichter, H.; Benayoun, M.: E-tree: emotionally driven augmented reality art. In Proc. ACM Multimedia, pages 945-948, Vancouver, BC, Canada, 2008. ACM.
  5. Hönig, F.; Wagner, J.; Batliner, A.; Nöth, E.: Classification of user states with physiological signals: On-line generic features vs. specialized feature sets. In Proc. of the 17th European Signal Processing Conference (EUSIPCO-2009), 2009.
  6. Jacucci, G. G.; Spagnolli, A.; Chalambalakis, A.; Morrison, A.; Liikkanen, L.; Roveda, S.; Bertoncini. M.: Bodily explorations in space: Social experience of a multimodal art installation. In Proc. of the twelfth IFIP conference on Human-Computer Interaction: Interact, 2009.
  7. Kim, J.; André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. and Machine Intell., 30(12):2067-2083, 2008.
  8. Kim, J.; Ragnoni, A.; Biancat, J.: In-Vehicle Monitoring Of Affective Symptoms for Diabetic Drivers. In Proc. of the Int. Conf. on Health Informatics (HEALTHINF), 2010. [in press]
  9. Vogt, T.; André, E.; Bee, N.: A framework for online recognition of emotions from voice. In Proc. of Workshop on Perception and Interactive Technologies for Speech-Based Systems, Kloster Irsee, Germany, 2008.
  10. Wagner, J.; Kim, J.; André, E.: From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification. In Proc. IEEE ICME 2005, pp. 940-943, Amsterdam, 2005.
  11. Wagner, J.; André; E., Jung, F.: Smart sensor integration: A framework for multimodal emotion recognition in real-time. In Affective Computing and Intelligent Interaction (ACII 2009), 2009.
  12. Zeng, Z.; Pantic, M.; Roisman, G I.; Huang, T S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell., 31(1):39-58, 2009.
Download


Paper Citation


in Harvard Style

Wagner J., Jung F., Kim J., Vogt T. and André E. (2010). The Smart Sensor Integration Framework and its Application in EU Projects . In Proceedings of the 1st International Workshop on Bio-inspired Human-Machine Interfaces and Healthcare Applications - Volume 1: B-Interface, (BIOSTEC 2010) ISBN 978-989-674-020-7, pages 13-21. DOI: 10.5220/0002813900130021


in Bibtex Style

@conference{b-interface10,
author={Johannes Wagner and Frank Jung and Jonghwa Kim and Thurid Vogt and Elisabeth André},
title={The Smart Sensor Integration Framework and its Application in EU Projects},
booktitle={Proceedings of the 1st International Workshop on Bio-inspired Human-Machine Interfaces and Healthcare Applications - Volume 1: B-Interface, (BIOSTEC 2010)},
year={2010},
pages={13-21},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002813900130021},
isbn={978-989-674-020-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 1st International Workshop on Bio-inspired Human-Machine Interfaces and Healthcare Applications - Volume 1: B-Interface, (BIOSTEC 2010)
TI - The Smart Sensor Integration Framework and its Application in EU Projects
SN - 978-989-674-020-7
AU - Wagner J.
AU - Jung F.
AU - Kim J.
AU - Vogt T.
AU - André E.
PY - 2010
SP - 13
EP - 21
DO - 10.5220/0002813900130021