BIOSTORIES - Dynamic Multimedia Interfaces based on Automatic Real-time User Emotion Assessment

Vasco Vinhas, Eugénio Oliveira, Luís Paulo Reis

Abstract

BioStories is the outcome of a three and a half years research project focused in uniting affective and ubiquitous computing with context aware multimedia content generation and distribution. Its initial premise was based in the possibility of performing real-time automatic emotion assessment trough online biometric channels monitoring and use this information to design on-the-fly dynamic multimedia storylines emotionally adapted, so that end users would unconsciously be choosing the story flow. The emotion assessment process was based on dynamic fusion of biometric channels such as EEG, GSR, respiration rate and volume, skin temperature and heart rate on top of Russell’s circumplex model of affect. BioStories’ broad scope also allowed for some spin-off projects namely mouse control through EMG that resulted in a patented technology for alternative/inclusive interfaces. Exhaustive experiments showed 87% of success rate for emotion assessment in a dynamic tridimensional virtual environment with an immersiveness score of 4.2 out of 5. The success of the proposed approach allows the vision of its appliance in several domains such as virtual entertainment, videogames and cinema as well as direct marketing, digital TV and domotic appliances.

References

  1. António Gomes, Vasco Vinhas, Mouse Control Through Electromyography, in BIOSIGNALS 2008 - International Conference on Bio-inspired Systems and Signal Processing, pp. 371-376, 2008.
  2. Benovoy, M., Cooperstock, J., Deitcher, J., Biosignals Analysis and its Application in a Performance Setting - Towards the Development of an Emotional-Imaging Generator, in Proceedings of the First Inte Conference on Biomedical Electronics and Devices, pp. 253-258, 2008.
  3. Egon L. van den Broek et al, Biosignals as an Advanced Man-Machine Interface, in BIOSTEC International Joint Conference on Biomedical Engineering Systems and Technologies, pp.15-24, 2009.
  4. Ekman P., Emotion in the Human Face, Cambridge University Press, pp. 39-55, 2005.
  5. Hua Wang, H. Prendinger, T. Igarashi, Communicating emotions in online chat using physiological sensors and animated text, in Conference on Human Factors in Computing System, pp 1171-1174, 2004.
  6. Jae-woo Chung, G. Scott Vercoe, The affective remixer: personalized music arranging, in Conference on Human Factors in Computing Systems, pp 393-398, 2006
  7. Jorge Teixeira, Vasco Vinhas, Eugénio Oliveira, Luís Paulo Reis, General-Purpose Emotion Assessment Testbed Based on Biometric Information, KES IIMSS - Intelligent Interactive Multimedia Systems and Services, pp.533-543, University of Piraeus, Greece, 2008.
  8. Jorge Teixeira, Vasco Vinhas, Eugénio Oliveira, Luís Paulo Reis, MultiChannel Emotion Assessment Framework - Gender and High-Frequency Electroencephalography as Key-Factors, in Proceedings of ICEIS 2008 - 10th International Conference on Enterprise Information Systems, pp. 331-334,, 2008.
  9. Jorge Teixeira, Vasco Vinhas, Eugénio Oliveira, Luís Paulo Reis, A New Approach to Emotion Assessment Based on Biometric Data, WI-IAT 7808 - IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, 9-12 Dec, Sydney, Australia, pp. 505-511, 2008.
  10. Katsis, C., Katertsidis N., Ganiatsas G. and Fotiadis D., Towards Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach, in IEEE Tran. on Systems, Man, and Cybernetics - Part A: Systems and Humans, Vol. 38, No 3, pp. 502-512, 2008.
  11. Kim, J., André, E., Multi-Channel BioSignal Analysis for Automatic Emotion Recognition, in Proceedings of the First International Conference on Biomedical Electronics and Devices, 2008.
  12. Picard, R. W., Vyzas, E., Healey, J., Toward Machine Emotional Intelligence: Analysis of Affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 23, Issue 10, pp.1175-1191, 2001.
  13. Russell, J. A., A Circumplex Model of Affect, in Journal of Personality and Social Psychology, No 39, pp 1161-1178, 1980.
  14. Russell, J. A., Evidence for a Three-Factor Theory of Emotions, in Journal of Research in Personality, Vol 11, pp 273-294, 1977.
  15. Stephen H. Fairclough, Fundamentals of Physiological Computing, in Interaction with Computers, Vol 21, Issue 1-2, pp. 133-145, 2009.
  16. Vasco Vinhas, Eugénio Oliveira, Luís Paulo Reis, Realtime Dynamic Multimedia Storyline Based on Online Audience Biometric Information, KES IIMSS - Intelligent Interactive Multimedia Systems and Services, pp.545-554, University of Piraeus, Greece, 2008.
  17. Vasco Vinhas, Eugénio Oliveira, Luís Paulo Reis, Dynamic Multimedia Content Delivery Based on Real-Time User Emotions - Multichannel Online Biosignals Towards Adaptative GUI and Content Delivery, in International Conference on Bio-inspired Systems and Signal Processing, pp.299-304, , 2009.
  18. Vasco Vinhas, Daniel Castro Silva, Eugénio Oliveira, Luís Paulo Reis, Dynamic Multimedia Environment Based On Real-Time User Emotion Assessment - Biometric User Data Towards Affective Immersive Environments, ICEIS 2009 - International Conference on Enterprise Information Systems, 2009.
Download


Paper Citation


in Harvard Style

Vinhas V., Oliveira E. and Reis L. (2010). BIOSTORIES - Dynamic Multimedia Interfaces based on Automatic Real-time User Emotion Assessment . In Proceedings of the 12th International Conference on Enterprise Information Systems - Volume 5: ICEIS, ISBN 978-989-8425-08-9, pages 21-29. DOI: 10.5220/0002866700210029


in Bibtex Style

@conference{iceis10,
author={Vasco Vinhas and Eugénio Oliveira and Luís Paulo Reis},
title={BIOSTORIES - Dynamic Multimedia Interfaces based on Automatic Real-time User Emotion Assessment},
booktitle={Proceedings of the 12th International Conference on Enterprise Information Systems - Volume 5: ICEIS,},
year={2010},
pages={21-29},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002866700210029},
isbn={978-989-8425-08-9},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Conference on Enterprise Information Systems - Volume 5: ICEIS,
TI - BIOSTORIES - Dynamic Multimedia Interfaces based on Automatic Real-time User Emotion Assessment
SN - 978-989-8425-08-9
AU - Vinhas V.
AU - Oliveira E.
AU - Reis L.
PY - 2010
SP - 21
EP - 29
DO - 10.5220/0002866700210029