Combined Framework for Real-time Head Pose Estimation using Facial Landmark Detection and Salient Feature Tracking

Jilliam María Díaz Barros, Frederic Garcia, Bruno Mirbach, Kiran Varanasi, Didier Stricker

2018

Abstract

This paper presents a novel approach to address the head pose estimation (HPE) problem in real world and demanding applications. We propose a new framework that combines the detection of facial landmarks with the tracking of salient features within the head region. That is, rigid facial landmarks are detected from a given face image, while at the same time, salient features are detected within the head region. The 3D coordinates of both set of features result from their intersection on a simple geometric head model (e.g., cylinder or ellipsoid). We then formulate the HPE problem as a perspective-n-point problem that we separately solve by minimizing the reprojection error of each 3D features set and their corresponding facial or salient features in the next face image. The resulting head pose estimations are then combined using Kalman Filter, which allows us to take advantage of the high accuracy when using facial landmarks while enabling us to handle extreme head poses by using salient features. Results are comparable to those from the related literature, with the advantage of being robust under real world situations that might not be covered in the evaluated datasets.

Paper Citation

in Harvard Style

Barros J., Garcia F., Mirbach B., Varanasi K. and Stricker D. (2018). Combined Framework for Real-time Head Pose Estimation using Facial Landmark Detection and Salient Feature Tracking. In Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2018) - Volume 5: VISAPP; ISBN 978-989-758-290-5, SciTePress, pages 123-133. DOI: 10.5220/0006628701230133

in Bibtex Style

@conference{visapp18,
author={Jilliam María Díaz Barros and Frederic Garcia and Bruno Mirbach and Kiran Varanasi and Didier Stricker},
title={Combined Framework for Real-time Head Pose Estimation using Facial Landmark Detection and Salient Feature Tracking},
booktitle={Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2018) - Volume 5: VISAPP},
year={2018},
pages={123-133},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006628701230133},
isbn={978-989-758-290-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2018) - Volume 5: VISAPP
TI - Combined Framework for Real-time Head Pose Estimation using Facial Landmark Detection and Salient Feature Tracking
SN - 978-989-758-290-5
AU - Barros J.
AU - Garcia F.
AU - Mirbach B.
AU - Varanasi K.
AU - Stricker D.
PY - 2018
SP - 123
EP - 133
DO - 10.5220/0006628701230133
PB - SciTePress