Facial Affect Recognition for Cognitive-behavioural Therapy

Dimitri Konstantas, Fernando Fernández–Aranda, Juan J. Santamaría, Maher Ben Moussa, Nadia Magnenat-Thalmann, Susana Jiménez-Murcia

2012

Abstract

In this paper we present facial affect recognition specifically designed and developed for clinical use in Cognitive-Behavioural Therapy. We describe our hybrid approach for facial affect recognition that combines geometric and appearance based features of the face. For geometric features we have developed a robust Active Shape Model based facial point tracking system for real-time use in clinical environments. Gabor wavelets are employed to extract the appearance-based features from appearance changes of the face skin. Further, we present an evaluation of our approach and we describe the systems integration with serious games that are being used for mental disorder therapy.

References

  1. Moore, R., Lopes, J., 1999. Paper templates. In TEMPLATE'06, 1st International Conference on Template Production. SciTePress.
  2. Smith, J., 1998. The book, The publishing company. London, 2nd edition.
  3. Fernandez-Aranda, F., Nunez, A. et al. 2008. InternetBased Cognitive-Behavioral Therapy for Bulimia Nervosa: A Controlled Study. Cyberpsychol Behav. 12.
  4. Jimenez-Murcia S, A.-M. E., Granero R, et al.,2007, Cognitive-behavioral group treatment for pathological gambling: analysis of effectiveness and predictors of therapy outcome. Psychotherapy Research, 17(5), 544 - 552.
  5. Alvarez-Moya, E. M., Jimenez-Murcia, S., Moragas, L., Gomez-Pena, M., Aymami, M. N., Ochoa, C., et al., 2009, Executive functioning among female pathological gambling and bulimia nervosa patients: preliminary findings. J Int Neuropsychol Soc, 15(2), 302-306.
  6. Fernandez-Aranda, F., Jimenez-Murcia, S., AlvarezMoya, E. M., Granero, R., Vallejo, J., & Bulik, C. M., 2006, Impulse control disorders in eating disorders: clinical and therapeutic implications. Compr Psychiatry, 47(6), 482-488.
  7. Viola, P. and Jones, M. J., 2004, Robust Real-Time Face Detection, International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154
  8. Cohen, I., Sebe, N., Garg, A., Chen, L. S., and Huanga, T. S., 2003, Facial expression recognition from video sequences: temporal and static modeling, Computer Vision and Image Understanding, vol. 91, no. 1-2, pp. 160-187
  9. Bartlett, M. S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., and Movellan, J., 2006, Fully Automatic Facial Action Recognition in Spontaneous Behavior, in 7th International Conference on Automatic Face and Gesture Recognition (FGR06), pp. 223-230
  10. Tian, Y.-li, Kanade, T., and Cohn, J. F., 2005, Facial Expression Analysis, Handbook of Face Recognition, vol. 3, no. 5, pp. 247-275
  11. Patras, I. abd Pantic, M., 2005, Tracking deformable motion, International Conference on Systems, Man and Cybernetics, IEEE, vol.2, no., pp.1066-1071
  12. Cootes, T. and Taylor, C., 2004, Statistical models of appearance for computer vision, Technical report, Imaging Science and Biomedical Engineering, University of Manchester
  13. Matthews, I. and Baker, S., 2004, Active appearance models revisited, International Journal of Computer Vision, 60(2):135-164
  14. Stegmann, M. B., 2000, Active Appearance Models: Theory, Extensions and Cases
  15. Ford, G., 2002, Fully automatic coding of basic expressions from video, Technical Report INCMPLab-TR-2002.03, Machine Perception Lab, Institute for Neural Computation, University of California, San Diego
  16. Cohn, J. F., Reed, L. I., Ambadar, Z., and Moriyama, T., 2004, Automatic analysis and recognition of brow actions and head motion in spontaneous facial behavior, in IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583), vol. 1, pp. 610-616.
  17. Vanderlinden, J., Dalle Grave, R., Fernandez, F., Vandereycken, W., Pieters, G., & Noorduin, C., 2004. Which factors do provoke binge eating? An exploratory study in eating disorder patients. Eat Weight Disord, 9(4), 300-305
  18. Ledgerwood, D. M., & Petry, N. M., 2006, Psychological experience of gambling and subtypes of pathological gamblers. Psychiatry Res, 144(1), 17-27.
  19. Castrillón, M., Déniz, O., Guerra, C., and Hernández, M., 2007, ENCARA2: Real-time detection of multiple faces at different resolutions in video streams, Journal of Visual Communication and Image Representation, vol. 18, no. 2, pp. 130-140
  20. Jia, P., 2010, 2D Statistical Models, Technical Report of Vision Open Working Group, 2st Edition, Oct 21
  21. Jia, P., Audio-visual based HMI for an Intelligent Wheelchair. PhD thesis, University of Essex, 2010
  22. Kostoulas T. et al., 2010, The PlayMancer Database: A Multimodal Affect Database in Support of Research and Development Activities in Serious Game Environment, in Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC 2010), Valetta, Malta
  23. Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I., 2010, The Extended CohnKanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression, in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, pp. 94-101
  24. Jiménez-Murcia S, Fernández-Aranda F, Kalapanidas E, Konstantas D, Ganchev T, Kocsis O, Lam T, Santamaría JJ, Raguin T, Breiteneder C, Kaufmann H, Davarakis C., 2009, Playmancer project: a serious videogame as an additional therapy tool for eating and impulse control disorders. Stud Health Technol Inform,144:163-6.
  25. Fernández-Aranda F, Jiménez-Murcia S, Santamaría JJ, Gunnard K, Soto A, Kalapanidas E, Bults R, Davarakis C, Ganchev T, Granero R, Konstantas D, Kostoulas T, Lam T, Lucas M, Masuet-aumatell C, Ben Moussa M, Nielsen J & Penelo E., 2012,Video games as a complementary therapy tool in mental disorders: PlayMancer, a European multicentre study, Journal of Mental Health
Download


Paper Citation


in Harvard Style

Moussa M., Magnenat-Thalmann N., Konstantas D., Fernández–Aranda F., Santamaría J. and Jiménez-Murcia S. (2012). Facial Affect Recognition for Cognitive-behavioural Therapy . In Proceedings of the Sixth International Symposium on e-Health Services and Technologies and the Third International Conference on Green IT Solutions - Volume 1: EHST, ISBN 978-989-8565-27-3, pages 64-68. DOI: 10.5220/0004474200640068


in Bibtex Style

@conference{ehst12,
author={Maher Ben Moussa and Nadia Magnenat-Thalmann and Dimitri Konstantas and Fernando Fernández–Aranda and Juan J. Santamaría and Susana Jiménez-Murcia},
title={Facial Affect Recognition for Cognitive-behavioural Therapy},
booktitle={Proceedings of the Sixth International Symposium on e-Health Services and Technologies and the Third International Conference on Green IT Solutions - Volume 1: EHST,},
year={2012},
pages={64-68},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004474200640068},
isbn={978-989-8565-27-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Sixth International Symposium on e-Health Services and Technologies and the Third International Conference on Green IT Solutions - Volume 1: EHST,
TI - Facial Affect Recognition for Cognitive-behavioural Therapy
SN - 978-989-8565-27-3
AU - Moussa M.
AU - Magnenat-Thalmann N.
AU - Konstantas D.
AU - Fernández–Aranda F.
AU - Santamaría J.
AU - Jiménez-Murcia S.
PY - 2012
SP - 64
EP - 68
DO - 10.5220/0004474200640068