REAL-TIME LIPTRACKING FOR SYNTHETIC FACE ANIMATION WITH FEEDBACK LOOP

Franck Luthon, Brice Beaumesnil

Abstract

This article deals with facial segmentation and liptracking with feedback control for real-time animation of a synthetic 3D face model. Straightforward approaches consist in two successive steps: video analysis then synthesis. Our approach departs from the previous ones in that we build a global analysis/synthesis processing loop, where the image analysis needs the 3D synthesis and conversely. A first facial segmentation is computed according to which the 3D face model is positionned. Then the feedback loop, implemented from the 3D animated model back to the input pixel segmentation algorithm, helps to correct some (few) control points that were badly tracked, which are detected by measuring the vertical distance between lip contour and corresponding 3D face model. When this distance is too big, we re-enter into the image segmentation process and zoom-in inside a few regions of interest (ROI) where the algorithm is run again, with a new set of tuning parameters better suited to the pixel neighborhood context. In that way, the face segmentation is refined in order to extract more precise parameters. This approach is inspired from control theory with closed-loop systems. The contribution of the paper is to use simple image processing techniques, but to improve segmentation through the feedback loop. Results show that real-time and robust performances are achievable under real-world conditions, which are two key issues for face and lip tracking applications.

References

  1. Batur, A. U. and Hayes, M. H. (2005). Adaptive active appearance models. IEEE Trans. on Image Processing, 14(11):1707-1721.
  2. Benoˆit, C., Lallouache, T., Mohamadi, T., and Abry, C. (1992). A set of French visemes for visual speech synthesis. In Bailly, G., Benoˆit, C., and Sawallis, T., editors, Talking Machines: Theories, Models and Designs, pages 485-504, Amsterdam, North-Holland. Elsevier Science Publishers B.V.
  3. Cootes, T., Edwards, G., and Taylor, C. (1998). Active appearance models. In European Conference on Computer Vision, ECCV, volume 2, pages 484-498. Springer-Verlag.
  4. Erdem, C. E., Sankur, B., and Tekalp, A. M. (2004). Performance measures for video object segmentation and tracking. IEEE Trans. on Image Processing, 13(7):937-951.
  5. Erdem, C. E., Tekalp, A. M., and Sankur, B. (2003). Video object tracking with feedback of performance measures. IEEE Trans. on Circuits and Systems for Video Technology, 13(4):310-324.
  6. Fu, Y., Erdem, A. T., and Tekalp, A. M. (2000). Tracking visible boundary of objects using occlusion adaptive motion snake. IEEE Trans. on Image Processing, 9(12):2051-2060.
  7. Kass, M., Witkin, A., and Terzopoulos, D. (1987). Snakes: Active contour models. International Journal of Computer Vision, pages 321-331.
  8. Liévin, M., Delmas, P., Coulon, P. Y., Luthon, F., and Fristot, V. (1999). Automatic lip tracking: Bayesian segmentation and active contours in a cooperative scheme. In IEEE Int. Conf. on Multimedia Computing and Systems (ICMCS'99), pages 691-696, Firenze, Italy. Vol. 1.
  9. Liévin, M. and Luthon, F. (2004). Nonlinear color space and spatiotemporal MRF for hierarchical segmentation of face features in video. IEEE Transactions on Image Processing, 13(1):63-71.
  10. Mirmehdi, M., Palmer, P. L., Kitler, J., and Dabis, H. (1999). Feedback control strategies for object recognition. IEEE Trans. on Image Processing, 8(8):1084- 1101.
  11. Pichler, O., Teuner, A., and Hosticka, B. J. (1998). An unsupervised texture segmentation algorithm with feature space reduction and knowledge feedback. IEEE Trans. on Image Processing, 7(1):53-61.
Download


Paper Citation


in Harvard Style

Luthon F. and Beaumesnil B. (2006). REAL-TIME LIPTRACKING FOR SYNTHETIC FACE ANIMATION WITH FEEDBACK LOOP . In Proceedings of the First International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, ISBN 972-8865-40-6, pages 402-407. DOI: 10.5220/0001367604020407


in Bibtex Style

@conference{visapp06,
author={Franck Luthon and Brice Beaumesnil},
title={REAL-TIME LIPTRACKING FOR SYNTHETIC FACE ANIMATION WITH FEEDBACK LOOP},
booktitle={Proceedings of the First International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP,},
year={2006},
pages={402-407},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001367604020407},
isbn={972-8865-40-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the First International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP,
TI - REAL-TIME LIPTRACKING FOR SYNTHETIC FACE ANIMATION WITH FEEDBACK LOOP
SN - 972-8865-40-6
AU - Luthon F.
AU - Beaumesnil B.
PY - 2006
SP - 402
EP - 407
DO - 10.5220/0001367604020407