Real Time Eye Gaze Tracking System using CNN-based Facial Features for Human Attention Measurement

Oliver Lorenz, Ulrike Thomas

2019

Abstract

Understanding human attentions in various interactive scenarios is an important task for human-robot collaboration. Human communication with robots includes intuitive nonverbal behaviour body postures and gestures. Multiple communication channels can be used to obtain a understandable interaction between humans and robots. Usually, humans communicate in the direction of eye gaze and head orientation. In this paper, a new tracking system based on two cascaded CNNs is presented for eye gaze and head orientation tracking and enables robots to measure the willingness of humans to interact via eye contacts and eye gaze orientations. Based on the two consecutively cascaded CNNs, facial features are recognised, at first in the face and then in the regions of eyes. These features are detected by a geometrical method and deliver the orientation of the head to determine eye gaze direction. Our method allows to distinguish between front faces and side faces. With a consecutive approach for each condition, the eye gaze is also detected under extreme situations. The applied CNNs have been trained by many different datasets and annotations, thereby the reliability and accuracy of the here introduced tracking system is improved and outperforms previous detection algorithm. Our system is applied on commonly used RGB-D images and implemented on a GPU to achieve real time performance. The evaluation shows that our approach operates accurately in challenging dynamic environments.

Paper Citation

in Harvard Style

Lorenz O. and Thomas U. (2019). Real Time Eye Gaze Tracking System using CNN-based Facial Features for Human Attention Measurement. In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 5: VISAPP; ISBN 978-989-758-354-4, SciTePress, pages 598-606. DOI: 10.5220/0007565305980606

in Bibtex Style

@conference{visapp19,
author={Oliver Lorenz and Ulrike Thomas},
title={Real Time Eye Gaze Tracking System using CNN-based Facial Features for Human Attention Measurement},
booktitle={Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 5: VISAPP},
year={2019},
pages={598-606},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007565305980606},
isbn={978-989-758-354-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 5: VISAPP
TI - Real Time Eye Gaze Tracking System using CNN-based Facial Features for Human Attention Measurement
SN - 978-989-758-354-4
AU - Lorenz O.
AU - Thomas U.
PY - 2019
SP - 598
EP - 606
DO - 10.5220/0007565305980606
PB - SciTePress