Constructing Facial Expression Log from Video Sequences using Face Quality Assessment

Mohammad A. Haque, Kamal Nasrollahi, Thomas B. Moeslund

2014

Abstract

Facial expression logs from long video sequences effectively provide the opportunity to analyse facial expression changes for medical diagnosis, behaviour analysis, and smart home management. Generating facial expression log involves expression recognition from each frame of a video. However, expression recognition performance greatly depends on the quality of the face image in the video. When a facial video is captured, it can be subjected to problems like low resolution, pose variation, low brightness, and motion blur. Thus, this paper proposes a system for constructing facial expression log by employing a face quality assessment method and investigates its influence on the representations of facial expression logs of long video sequences. A framework is defined to incorporate face quality assessment with facial expression recognition and logging system. While assessing the face quality a face-completeness metric is used along with some other state-of-the-art metrics. Instead of discarding all of the low quality faces from a video sequence, a windowing approach has been applied to select best quality faces in regular intervals. Experimental results show a good agreement between the expression logs generated from all face frames and the expression logs generated by selecting best faces in regular intervals.

References

  1. Ahmed, E. L., Rara, H., Farag, A., and Womble, P., 2012. “Face Detection at a distance using saliency maps,” IEEE Conf. on Computer Vision and Pattern Recogntion Workshops, pp. 31-36.
  2. Axnick, K., Jarvis, R., and Ng, K. C., 2009. “Using Face Quality Ratings to Improve Real-Time Face Recognition,” Proc. of the 3rd Pacific Rim Symposium on Advances in Image and Video Technology, pp. 13- 24.
  3. Bagdanov, A. D., and Bimbo, A. D., 2012. “Posterity Logging of Face Imagery for Video Surveillance,” IEEE MultiMedia, vol. 19, no. 4, pp. 48-59.
  4. Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., and Movellan, J., 2006. “Fully Automatic Facial Action Recognition in Spontaneous Behavior,” 7th Int. COnf. on Automatic Face and Gesture Recogntion, pp. 223-230.
  5. Bonner, M. J. et al., 2008. “Social Functioning and Facial Expression Recognition in Survivors of Pediatric Brain Tumors”, Journal of Pediatric Psychology, vol. 33, no. 10, pp. 1142-1152, 2008.
  6. Briechle, K., and Hanebeck, U. D., 2001. “Template Matching using Fast Normalized Cross Correlation,” Proc. of SPIE Aero Sense Symposium, vol. 43-87, pp. 1-8.
  7. Busso, C., Narayanan, S. S., 2007. “Interrelation between Speech and Facial Gestures in Emotional Utterances: A single subject study”, IEEE Trans. On Audio, Speech and Language Processing, pp. 1-16.
  8. Cheng, X., Lakemond, R., Fookes, C., and Sridharan, S., 2012. “Efficient Real-Time Face Detection For High Resolution Surveillance Applications,” Proc. Of the 6th Int. Conf. on Signal Processing and Communication Systems, pp.1-6.
  9. Cohn, J., Kreuz, T., Yang, Y., Nguyen, M., Padilla, M., Zhou, F., and Fernando, D., 2009, “Detecting depression from facial action and vocal prosody,” Proc. of the Int. Conf. on Affective Computing and Intelligent Interaction.
  10. Corcoran, P., Steingerg, E., Bigioi, P., and Brimbarean, A., 2007. “Real-Time Face Tracking in a Digital Image Acquisition Device,” European Patent No. EP 2 052 349 B1, pp. 25.
  11. Dhillon, P. S., 2009. “Robust Real-Time Face Tracking Using an Active Camera,” Advances in Intellignet and Soft Computing: Computational Intelligences in Security for Information Systems, vol. 63, pp. 179-186.
  12. Dinh, T., Yu, Q., and Medioni, G., 2009. “Real Time Tracking using an Active Pan-Tilt-Zoom network Camera,” Proc. of the Int. Conf. on Intelligent Robots and Systems, pp. 3786-3793.
  13. Dinh, T. B., Vo, N., and Medioni, G., 2011. “High Resolution Face Sequences from a PTZ Network Camera,” Proc. of the IEEE Int. Conf. on Automatic Face & Gesture Recognition and Workshops, pp. 531- 538.
  14. Dong, G., and Lu, S., 2010. “The relation of expression recognition and affective experience in facial expression processing: an event-related potential study,” Psychology Research and Behavior Management, vol. 3, pp. 65-74.
  15. Doody, R. S., Stevens, J. C., Beck, C., et al. 2013, “Practice parameter: Management of dementia (an evidence-based review): Report of the quality standards subcommittee of the American Academy of Neurology,” Neurology, pp. 1-15.
  16. Fang, S. et al., 2008. “Automated Diagnosis of Fetal Alcohol Syndrome Using 3D Facial Image Analysis,” Orthodontics & Craniofacial Research, vol. 11, no. 3, pp. 162-171.
  17. Fraunhofer IIS, 2013. “Sophisticated High-speed Object Recognition Engine,” www.iis.fraunhofer.de/shore.
  18. Jun-Su, J., and Jong-Hwan, K., “Fast and Robust face Detection using Evolutionary Prunning,” IEEE Trans. On Evolutionary Computation, vol. 12, no. 5, pp. 562- 571, 2008.
  19. Kanade, T., Cohn, J., Tian, Y. L., 2000. “Comprehensive database for facial expression analysis,” Proc. of the Int. Conf. on Face and Gesture Recognition, pp. 46- 53.
  20. Kublbeck, C., and Ernst, A., 2006. “Face detection and tracking in video sequences using the modified census transformation,” Image and Vision Computing, vol. 24, no. 6, pp. 564-572.
  21. Lee, Y. B., and Lee, S., 2011. “Robust Face Detection Based on Knowledge Directed Specification of Bottom-Up Saliency,” ETRI Journal, vol. 33, no. 4, pp. 600-610.
  22. Mohammad, A. H., Nasrollahi, K., and Moeslund, T. B., 2013. “Real-Time Acquisition of High Quality Face Sequences From an Active Pan-Tilt-Zoom Camera,” Proc. of the Int. Conf. on Advanced Video and Surveillance Systems, pp. 1-6.
  23. Mustafah, Y. M., Shan, T., Azman, A. W., Bigdeli, A., and Lovell, B. C., 2007. “Real-Time Face Detection and Tracking for High Resolution Smart Camera System,” Proc. Of the 9th Biennial Conf. on Digital Image Computing Techniques and Applications, pp. 387-393.
  24. Mustafah, Y. M., Bigdeli, A., Azman, A. W., and Lovell, B. C., 2009. “Face Detection System Design for RealTime High Resolution Smart Camera,” Proc. Of the 3rd ACM/IEEE Int. Conf. on Distributed Smart Cameras, pp. 1-6.
  25. Nasrollahi, K., and Moeslund, T. B., 2008. “Face Quality Assessment System in Video Sequences,” 1st European Workshop on Biometrics and Identity Management, Springer-LNCS, vol. 5372, pp. 10-18.
  26. Nasrollahi, K., and Moeslund, T. B., 2009. “Complete face logs for video sequences using face quality measures,” IET Signal Porcessing, vol. 3, no. 4. Pp. 289-300, DOI 10.1049/iet-spr.2008.0172.
  27. Russell, A. J., and Fehr, B., 1987, “Relativity in the Perception of Emotion in Facial Expressions” Journal of Experimental Psychology, vol. 116, no.3, pp. 223- 237.
  28. Suwa, M., Sugie, N., and Fujimora, K., 1978. “A preliminary note on pattern recognition of human emotional expression,” Proc. of the Int. Joint Conf. on Pattern Recognition, pp. 408-410.
  29. Tian, Y., Kanade, T., and Cohn, J. F., 2011. “Facial Experssion Recognition,” Handbook of Face Recognition, Chapter 19, pp. 487-519.
  30. Viola, P., and Jones, M., 2001 “Robust real-time face detection,” Proc. Of the 8th IEEE Int. Conf. on Computer Vision, pp. 747.
  31. Wen, Z., and Huang, T., 2003. “Capturing subtle facial motions in 3D face tracking,” Proc. of the Int. Conf. on Computer Vision.
  32. Wong, Y., Chen, S., Mau, S., Sanderson, C., and Lovell, B. C., 2011. “Patch-based Probabilistic Image Quality Assessment for Face Selection and Improved Videobased Face Recognition,” Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition Workshops, pp. 74-81.
Download


Paper Citation


in Harvard Style

A. Haque M., Nasrollahi K. and B. Moeslund T. (2014). Constructing Facial Expression Log from Video Sequences using Face Quality Assessment . In Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2014) ISBN 978-989-758-004-8, pages 517-525. DOI: 10.5220/0004730105170525


in Bibtex Style

@conference{visapp14,
author={Mohammad A. Haque and Kamal Nasrollahi and Thomas B. Moeslund},
title={Constructing Facial Expression Log from Video Sequences using Face Quality Assessment},
booktitle={Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2014)},
year={2014},
pages={517-525},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004730105170525},
isbn={978-989-758-004-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2014)
TI - Constructing Facial Expression Log from Video Sequences using Face Quality Assessment
SN - 978-989-758-004-8
AU - A. Haque M.
AU - Nasrollahi K.
AU - B. Moeslund T.
PY - 2014
SP - 517
EP - 525
DO - 10.5220/0004730105170525