FACIAL ANIMATION WITH MOTION CAPTURE BASED ON SURFACE BLENDING

Lijia Zhu, Won-Sook Lee

Abstract

This paper proposes a methodology to reconstruct 3D facial expressions with motion capture data. Feature-point based facial animation provides easy control of expression usually by moving surface points using mathematical deformation. However it does not support the high quality of surface animation of the face where the feature points are not present. In this paper, we focus on animating a 3D facial model using only feature points, but keeping the high quality animation by using an expression databank obtained from surface scanning. Firstly, a facial expression databank is prepared by processing raw laser-scanned human face data with a consistent parameterization technique. Secondly, sparse motion capture data is obtained using an optical tracking system. Thirdly, guided by the captured MPEG-4 feature point motions, we find the corresponding surface information in the existing examples in the databank by linear combination of them. The optimized blending weights are obtained implicitly by Genetic Algorithms (GA). Finally, the surface blending result is retargeted into the performer’s neutral facial mesh. Consequently, motions of the facial surface points are reconstructed.

References

  1. Busso, C., Deng, Z., Lopes, Yildirim, S., Bulut, M., Lee, C.M., Kazemzadeh, A., Lee, S., Neumann, U., Narayanan, S., 2004. Analysis of emotion recognition using facial expressions, speech and multimodal information. In Proc. of ACM 6th International Conference on Multimodal Interfaces (ICMI 2004). p.205-211. State College, PA.
  2. Chai, J., Xiao, J., Hodgins, J., 2003. Vision-based control of 3D facial animation. In Proc. of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer animation. San Diego, California.
  3. Guenter, B., Grimm, C., Wood, D., Malvar, H., Pighin, F., 1998. Making faces. In Proc. SIGGRAPH 1998. p. 55-66.
  4. Jeong, W.K., Kähler, K., Haber, J., Seidel, H.P., 2002. Automatic generation of subdivision surface head models from point cloud data. In Proc. Graphics Interface. p. 181-188.
  5. Kähler, K., Haber, J., Yamauchi, H., Seidel, H.P., 2002. Head shop: generating animated head models with anatomical structure. In Proc of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, San Antonio, Texas. p. 55 - 63.
  6. Kouadio, C., Poulin, P., Lachapelle, P., 1998. Real-time facial animation based upon a bank of 3D facial expressions. In Proc. of the Computer Animation.
  7. Kshirsagar, S., Molet, T., Magnenat-Thalmann, N., 2001. Principal components of expressive speech animation. In Proc. of Computer Graphics International 2001. p. 38-44.
  8. Lee, W.-S., Soon, A., 2006. Facial shape and 3D skin. Computer Animation and Virtual Worlds (SCI), Vol. 17, Issue 3-4 (p. 501-512). July 2006, John Wiley.
  9. Pighin, F., Szeliski, R., Salesin, D.H., 1999. Resynthesizing facial animation through 3D modelbased tracking. In Proc. of International Conference on Computer Vision. p.143--150. Corfu, Greece.
  10. Terzopoulos, D., Waters, K., 1993. Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Trans. Pattern Analysis and Machine Intelligence. 15(6):569-579.
  11. Noh, J.Y., Neumann, U., 2001. Expression cloning. In Proc. of SIGGRAPH 2001. ACM Press. p. 277-288.
  12. Williams, L., 1990. Performance-driven facial animation. In Proc. of SIGGRAPH 1990. p. 235--242.
  13. Zhang, Y., Sim, T., Tan, C. L., 2004. Adaptation-based individualized face modeling for animation using displacement map. In Proc. of Computer Graphics International 2004 (Crete, Greece). p. 518-521.
  14. Zhu, L., Lee, W.-S., May 2006. Modeling and animating for the dense laser-scanned face in the low resolution level. In Proc. of the 17th IASTED International Conference on Modelling & Simulation 2006. Montreal, Quebec, Canada.
  15. Zhu, L., Lee, W.-S., July 2006. Facial expression via genetic algorithms, In Proc .of the 19th Annual Conference on Computer Animation and Social Agents (CASA) 2006. Geneva, Switzerland.
Download


Paper Citation


in Harvard Style

Zhu L. and Lee W. (2007). FACIAL ANIMATION WITH MOTION CAPTURE BASED ON SURFACE BLENDING . In Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP, ISBN 978-972-8865-72-6, pages 63-68. DOI: 10.5220/0002083700630068


in Bibtex Style

@conference{grapp07,
author={Lijia Zhu and Won-Sook Lee},
title={FACIAL ANIMATION WITH MOTION CAPTURE BASED ON SURFACE BLENDING},
booktitle={Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP,},
year={2007},
pages={63-68},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002083700630068},
isbn={978-972-8865-72-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP,
TI - FACIAL ANIMATION WITH MOTION CAPTURE BASED ON SURFACE BLENDING
SN - 978-972-8865-72-6
AU - Zhu L.
AU - Lee W.
PY - 2007
SP - 63
EP - 68
DO - 10.5220/0002083700630068