Mobile Tutoring System in Facial Expression Perception and Production for Children with Autism Spectrum Disorder

Sergey Anishchenko, Alexandra Sarelaynen, Konstantin Kalinin, Anastasia Popova, Natalia Malygina-Lastovka, Kira Mesnyankina

2017

Abstract

Children with autism spectrum disorder are impaired in their ability to produce and recognize facial expressions and unable to interpret the social meaning of facial cues. Human interventionists can effectively train autistic individuals on facial expressions perception and production, they may benefit even more from computer-based intervention. In this study the tablet PC application was developed for learning facial expression perception and production. It uses newly designed computer vision algorithm for facial expression analysis which allows to estimate if the posed expression correct or not and guide users. Clinical trial was done with participation of 19 volunteer subjects from 6 to 12 years old. It was shown that after intervention subjects skills in emotion recognition were improved. Ability to transfer newly developed skills to children’s everyday life was also investigated. Parent’s questioning performed in 6 months after the intervention demonstrates that 10 out of 19 children were able to recognize emotion and respectively change their behavior in everyday life.

References

  1. Adolphs, R., Sears, L., & Piven, J., 2001. Abnormal processing of social information from faces in autism. Journal of Cognitive Neuroscience, 13(2), 232-240.
  2. Cockburn, J., Bartlett, M., Tanaka, J., Movellan, J., Pierce, M., & Schultz, R., 2008. SmileMaze?: A Tutoring System in Real-Time Facial Expression Perception and Production in Children with Autism Spectrum Disorder. Computer, 978-986.
  3. DeQuinzio, J. A., Townsend, D. B., Sturmey, P., & Poulson, C. L., 2007. Generalized imitation of facial models by children with autism. Journal of Applied Behavior Analysis, 40(4), 755-9.
  4. Ekman, P., & Rosenberg, E. L., 2012. What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS).
  5. Gordon, I., Pierce, M. D., Bartlett, M. S., & Tanaka, J. W., 2014. Training facial expression production in children on the autism spectrum. Journal of Autism and Developmental Disorders, 44(10), 2486-2498.
  6. Grynszpan, O., Weiss, P. L. (Tamar), Perez-Diaz, F., & Gal, E., 2014. Innovative technology-based interventions for autism spectrum disorders: A metaanalysis. Autism, 18(4), 346-361.
  7. Hashemi, J., Tepper, M., Vallin Spina, T., Esler, A., Morellas, V., Papanikolopoulos, N., … Sapiro, G., 2014. Computer vision tools for low-cost and noninvasive measurement of autism-related behaviors in infants. Autism Research and Treatment.
  8. Heimann, M., Nelson, K. E., Tjus, T., & Gillberg, C., 1995. Increasing reading and communication skills in children with autism through an interactive multimedia computer program. Journal of Autism and Developmental Disorders, 25(5), 459-480.
  9. Jain, S., Tamersoy, B., Zhang, Y., Aggarwal, J. K., & Orvalho, V., 2012. An interactive game for teaching facial expressions to children with autism spectrum disorders. In 5th International Symposium on Communications Control and Signal Processing, ISCCSP 2012.
  10. Littlewort, G., Whitehill, J., Wu, T., Fasel, I., Frank, M., Movellan, J., & Bartlett, M., 2011. The Computer Expression Recognition Toolbox (CERT). IEEE International Conference on Automatic Face & Gesture Recognition.
  11. M. Cox, J. Nuevo, J. S. and S. L., 2013. CSIRO Face Analysis SDK. In AFGR.
  12. Moore, M., & Calvert, S., 2000. Brief report: Vocabulary acquisition for children with autism. Teacher or computer instruction. Journal of Autism and Developmental Disorders, 30(4), 359-362.
  13. Olszanowski, M., Pochwatko, G., Kuklinski, K., SciborRylski, M., Lewinski, P., & Ohme, R. K., 2015. Warsaw set of emotional facial expression pictures: A validation study of facial display photographs. Frontiers in Psychology, 6(JAN), 1-8.
  14. Wallace, F., & Ekman, P., 1983. EMFACS-7: Emotional facial action coding system. Unpublished Manuscript, University of California at San Francisco, 2(36).
Download


Paper Citation


in Harvard Style

Anishchenko S., Sarelaynen A., Kalinin K., Popova A., Malygina-Lastovka N. and Mesnyankina K. (2017). Mobile Tutoring System in Facial Expression Perception and Production for Children with Autism Spectrum Disorder . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017) ISBN 978-989-758-226-4, pages 319-324. DOI: 10.5220/0006146003190324


in Bibtex Style

@conference{visapp17,
author={Sergey Anishchenko and Alexandra Sarelaynen and Konstantin Kalinin and Anastasia Popova and Natalia Malygina-Lastovka and Kira Mesnyankina},
title={Mobile Tutoring System in Facial Expression Perception and Production for Children with Autism Spectrum Disorder},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017)},
year={2017},
pages={319-324},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006146003190324},
isbn={978-989-758-226-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017)
TI - Mobile Tutoring System in Facial Expression Perception and Production for Children with Autism Spectrum Disorder
SN - 978-989-758-226-4
AU - Anishchenko S.
AU - Sarelaynen A.
AU - Kalinin K.
AU - Popova A.
AU - Malygina-Lastovka N.
AU - Mesnyankina K.
PY - 2017
SP - 319
EP - 324
DO - 10.5220/0006146003190324