FACE DETECTION AND TRACKING WITH 3D PGA CLM

Meng Yu, Bernard Tiddeman

Abstract

In this paper we describe a system for facial feature detection and tracking using a 3D extension of the Constrained Local Model (CLM) (Cristinacce and Cootes, 2006) algorithm. The use of a 3D shape model allows improved tracking through large head rotations. CLM uses a shape and texture appearance model to generate a set of region template detectors. A search is then performed in the global pose / shape space using these detectors. The proposed extension uses multiple appearance models from different viewpoints and a single 3D shape model built using Principal Geodesic Analysis (PGA) (Fletcher et al., 2004) instead of direct Principal Components Analysis (PCA). During fitting or tracking the current estimate of pose is used to select the appropriate appearance model. We demonstrate our results by fitting the model to image sequences with large head rotations. The results show that the proposed multi-view 3D CLM algorithm using PGA improves the performance of the algorithm using PCA for tracking faces in videos with large out-of-plane head rotations.

References

  1. Ahlberg, J. (2001). Using the active appearance algorithm for face and facial feature tracking. In International Conference on Computer Vision Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-time Systems, pages 68-72.
  2. Baker, S. and Matthews, I. (2001). Equivalence and efficiency of image alignment algorithms. In IEEE IEEE Transactions on Computer Vision and Pattern Recognition, pages 1090-1097.
  3. Baker, S. and Matthews, I. (2002). Lucas-kanade 20 years on: A unifying framework: Part 1. Technical Report CMU-RI-TR-02-16, Robotics Institute, University of Carnegie Mellon, Pittsburgh, PA.
  4. Blanz, V. and Vetter, T. (1999). A morphable model for the synthesis of 3d faces. In Computer graphics, annual conference series (SIG-GRAPH), pages 187-194.
  5. Brand, M. (2001). Morphable 3d models from video. In IEEE computer society conference on computer vision and pattern recognition, volume 2, pages 456-463.
  6. Chen, J. and Tiddeman, B. P. (2008). Multi-cue facial feature detection and tracking. In International Conference on Image and Signal Processing, pages 356-367.
  7. Cootes, T. F., Edwards, G. J., and Taylor, C. J. (2001). Active appearance models. In IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 681- 685.
  8. Cootes, T. F. and Kittipanyangam, P. (2002). Comparing variations on the active appearance model algorithm. In British Machine Vision Conference, volume 2, pages 837-846.
  9. Cootes, T. F. and Taylor, C. J. (2001). Statistical models of appearance for medical image analysis and computer vision. In SPIE Medical Imaging, pages 236-248.
  10. Cootes, T. F., Taylor, C. J., Cooper, D. H., and Graham, J. (1995). Active shape models - their training and application. Computer Vision and Image Understanding, 61:38-59.
  11. Cootes, T. F., Walker, K., and Taylor, C. (2000). View-based active appearance models. In IEEE International Conference on Automatic Face and Gesture Recognition, pages 227-232, Washington, DC, USA. IEEE Computer Society.
  12. Cristinacce, D. and Cootes, T. (2006). Feature detection and tracking with constrained local models. In British Machine Vision Conference, volume 3, pages 929-938.
  13. Faggian, N., Romdhani, S., Sherrah, J., and Paplinski, A. (2005). Color active appearance model analysis using a 3d morphable model. In Digital Image Computing on Techniques and Applications, page 59, Washington, DC, USA. IEEE Computer Society.
  14. Fletcher, P. T., Lu, C., Pizer, S. M., and Joshi, S. (2004). Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE transactions on medical imaging, 23:995-1005.
  15. Gross, R., Matthews, I., and Baker, S. (2005). Generic vs. person specific active appearance models. Image and Vision Computing, 23(11):1080-1093.
  16. Hager, G. D. and Belhumeur, P. N. (1998). Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20:1025-1039.
  17. Jones, M. J. and Poggio, T. (1998). Multidimensional morphable models: A framework for representing and matching object classes. In International Journal of Computer Vision, volume 29, pages 107-131. Springer Netherlands.
  18. Koterba, S., Baker, S., Matthews, I., Hu, C., Xiao, J., Cohn, J., and Kanade, T. (2005). Multi-view aam fitting and camera calibration. In IEEE International Conference on Computer Vision, pages 511-518, Washington, DC, USA. IEEE Computer Society.
  19. Lanitis, A., Taylor, C. J., and Cootes, T. F. (1997). Automatic interpretation and codeing of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):742-756.
  20. Lucas, B. D. and Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In International Joint Conference on Artificial Intelligence, pages 674-679.
  21. Matthews, I. and Baker, S. (2004). Active appearance models revisited. International Journal of Computer Vision, 60:135-164.
  22. Mitchell, S. C., Lelieveldt, B. P. F., Geest, R. J., Bosch, J. G., Reiber, J. H. C., and Sonka, M. (2001). Multistage hybrid active appearance model matching: Segmentation of left and right ventricles in cardiac mr images. In IEEE Transanctions on Medical Image, volume 20, pages 415-423.
  23. Nelder, J. A. and Mead, R. (1965). A simplex method for function minimization. Computer Journal, 7:308- 313.
  24. Paquet, U. (2009). Convexity and bayesian constrained local models. IEEE Transactions on Computer Vision and Pattern Recognition, pages 1193-1199.
  25. Peyras, J., Bartoli, A., Mercier, H., and Dalle, P. (2007). Segmented aams improve person-independent face fitting. In British Machine Vision Conference.
  26. Peyras, J., Bartoli, A. J., and Khoualed, S. K. (2008). Pools of aams: Towards automatically fitting any face image. In British Machine Vision Conference.
  27. Pizarro, D., Peyras, J., and Bartoli, A. (2008). Lightinvariant fitting of active appearance models. In IEEE Conference on Computer Vision and Patter Recognition, pages 1-6.
  28. Press., W. H., Teukolsky., S. A., Vetterling, W. T., and Flannery, B. P. (2007). Numerical recipes - The art of scientific computing. Cambridge University Press.
  29. Ramnath, K., Koterba, S., Xiao, J., Hu, C. B., Matthews, I., Baker, S., Cohn, J. F., and Kanade, T. (2008). Multiview aam fitting and construction. In International Journal of Computer Vision, volume 76, pages 183- 204.
  30. Romdhani, S., Blanz, V., and Vetter, T. (2002). Face identification by fitting a 3d morphable model using linear shape and texture error functions. In European Conference on Computer Vision, pages 3-19.
  31. Shum, H. Y. and Szeliski, R. (2001). Panoramic vision: sensors, theory, and applications, chapter Construction of panoramic image mosaics with global and local alignment, pages 227-268. Springer-Verlag New York, Inc., Secaucus, NJ, USA.
  32. Stegmann, M. B. (2001). Object tracking using active appearance models. In Danish Conference Pattern Recognition and Image Analysis, volume 1, pages 54- 60.
  33. Tiddeman, B. P. and Chen, J. (2007). Correlated active appearance models. In IEEE Transactions on SignalImage Technology & Internet-Based Systems, pages 832-838.
  34. Vetter, T. and Poggio, T. (1997). Linear object classes and image synthesis from a single example image. In Pattern Analysis and Machine Intelligence, volume 19(7), pages 733-742.
  35. Wang, Y., Lucey, S., Cohn, J., and Saragih, J. M. (2007). Non-rigid face tracking with local appearance consistency constraint. In IEEE International Conference on Automatic Face and Gesture Recognition.
  36. Wang, Y., Lucey, S., and Cohn, J. F. (2008). Enforcing convexity for improved alignment with constrained local models. In IEEE Conference on Computer Vision and Pattern Recognition, volume Issue 23-28, pages 1-8.
  37. Xiao, J., Baker, S., Matthews, I., and Kanade, T. (2004). Real-time combined 2d+3d active appearance models. In the IEEE computer society conference on computer vision and pattern recognition, volume 2, pages 535- 542.
  38. Yu, M. and Tiddeman, B. P. (2010). Facial feature detection and tracking with a 3d constrained local model. Submited to International Conferences in Central Europe on Computer Graphics, Visualization and Computer Vision 2010.
  39. Zhang, Z., Liu, Z., Adler, D., Cohen, M. F., Hanson, E., and Shan, Y. (2004). Robust and rapid generation of animated faces from video images: A model-based modeling approach. International Journal of Computer Vision, 58(2):93-119.
  40. Zhou, Y., Zhang, W., Tang, X., and Shum, H. (2005). A bayesian mixture model for multi-view face alignment. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 741-746, Washington, DC, USA. IEEE Computer Society.
Download


Paper Citation


in Harvard Style

Yu M. and Tiddeman B. (2010). FACE DETECTION AND TRACKING WITH 3D PGA CLM . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2010) ISBN 978-989-674-029-0, pages 44-53. DOI: 10.5220/0002829800440053


in Bibtex Style

@conference{visapp10,
author={Meng Yu and Bernard Tiddeman},
title={FACE DETECTION AND TRACKING WITH 3D PGA CLM},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2010)},
year={2010},
pages={44-53},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002829800440053},
isbn={978-989-674-029-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2010)
TI - FACE DETECTION AND TRACKING WITH 3D PGA CLM
SN - 978-989-674-029-0
AU - Yu M.
AU - Tiddeman B.
PY - 2010
SP - 44
EP - 53
DO - 10.5220/0002829800440053