Authors:
Catarina Runa Miranda
and
Verónica Costa Orvalho
Affiliation:
Universidade do Porto, Portugal
Keyword(s):
Facial Motion Capture, Emotion and Expressions recognition, Virtual Reality.
Related
Ontology
Subjects/Areas/Topics:
Applications and Services
;
Computer Vision, Visualization and Computer Graphics
;
Enterprise Information Systems
;
Features Extraction
;
Human and Computer Interaction
;
Human-Computer Interaction
;
Image and Video Analysis
;
Motion, Tracking and Stereo Vision
;
Optical Flow and Motion Analyses
Abstract:
Humans rely on facial expressions to transmit information, like mood and intentions, usually not provided by
the verbal communication channels. The recent advances in Virtual Reality (VR) at consumer-level (Oculus
VR 2014) created a shift in the way we interact with each other and digital media. Today, we can enter a
virtual environment and communicate through a 3D character. Hence, to the reproduction of the users’ facial
expressions in VR scenarios, we need the on-the-fly animation of the embodied 3D characters. However,
current facial animation approaches with Motion Capture (MoCap) are disabled due to persistent partial occlusions
produced by the VR headsets. The unique solution available for this occlusion problem is not suitable
for consumer-level applications, depending on complex hardware and calibrations. In this work, we propose
consumer-level methods for facial MoCap under VR environments. We start by deploying an occlusions-support
method for generic facial MoCap systems.
Then, we extract facial features to create Random Forests
algorithms that accurately estimate emotions and movements in occluded facial regions. Through our novel
methods, MoCap approaches are able to track non-occluded facial movements and estimate movements in
occluded regions, without additional hardware or tedious calibrations. We deliver and validate solutions to
facilitate face-to-face communication through facial expressions in VR environments.
(More)