Authors:
Stéphanie Lefèvre
and
Jean-Marc Odobez
Affiliation:
Idiap Research Institute;Ecole Polytechnique Fédérale de Lausanne, Switzerland
Keyword(s):
3D Head tracking, Appearance models, Structural features, View-based learning, Facial expression.
Related
Ontology
Subjects/Areas/Topics:
Applications
;
Computer Vision, Visualization and Computer Graphics
;
Human-Computer Interaction
;
Image and Video Analysis
;
Methodologies and Methods
;
Model-Based Object Tracking in Image Sequences
;
Motion and Tracking
;
Motion, Tracking and Stereo Vision
;
Pattern Recognition
;
Physiological Computing Systems
;
Software Engineering
;
Video Analysis
Abstract:
In this paper we address the issue of joint estimation of head pose and facial actions. We propose a method that can robustly track both subtle and extreme movements by combining two types of features: structural features observed at characteristic points of the face, and intensity features sampled from the facial texture. To handle the processing of extreme poses, we propose two innovations. The first one is to extend the deformable 3D face model Candide so that we can collect appearance information from the head sides as well as from the face. The second and main one is to exploit a set of view-based templates learned online to model the head appearance. This allows us to handle the appearance variation problem, inherent to intensity features and accentuated by the coarse geometry of our 3D head model. Experiments on the Boston University Face Tracking dataset show that the method can track common head movements with an accuracy of 3.2º, outperforming s
ome state-of-the-art methods. More importantly, the ability of the system to robustly track natural/faked facial actions and challenging head movements is demonstrated on several long video sequences.
(More)