Automatic Analysis of In-the-Wild Mobile Eye-tracking Experiments using Object, Face and Person Detection

Stijn De Beugher, Geert Brône, Toon Goedemé

2014

Abstract

In this paper we present a novel method for the automatic analysis of mobile eye-tracking data in natural environments. Mobile eye-trackers generate large amounts of data, making manual analysis very time-consuming. Available solutions, such as marker-based analysis minimize the manual labour but require experimental control, making real-life experiments practically unfeasible. We present a novel method for processing this mobile eye-tracking data by applying object, face and person detection algorithms. Furthermore we present a temporal smoothing technique to improve the detection rate and we trained a new detection model for occluded person and face detections. This enables the analysis to be performed on the object level rather than the traditionally used coordinate level. We present speed and accuracy results of our novel detection scheme on challenging, large-scale real-life experiments.

References

  1. Alves, R., Lim, V., Niforatos, E., Chen, M., Karapanos, E., and Nunes, N. J. (2012). Augmenting customer journey maps with quantitative empirical data: a case on eeg and eye tracking.
  2. Bay, H., Tuytelaars, T., and Van Gool, L. (2006). Surf: Speeded up robust features. In ECCV, pages 404-417.
  3. Broˆne, G., Oben, B., and Feyaerts, K. (2010). Insight interaction. a multimodal and multifocal dialogue corpus.
  4. Broˆne, G., Oben, B., and Goedemé, T. (2011). Towards a more effective method for analyzing mobile eyetracking data: integrating gaze data with object recognition algorithms. In Proc. of PETMEI, pages 53-56.
  5. Calonder, M., Lepetit, V., Strecha, C., and Fua, P. (2010). Brief: binary robust independent elementary features. In ECCV, pages 778-792.
  6. Dalal, N. and Triggs, B. (2005). Histograms of oriented gradients for human detection. In CVPR, pages 886- 893.
  7. De Beugher, S., Broˆne, G., and Goedemé, T. (2012). Automatic analysis of eye-tracking data using object detection algorithms. PETMEI.
  8. Dollár, P., Tu, Z., Perona, P., and Belongie, S. (2009). Integral channel features. In BMVC.
  9. Dollár, P., Wojek, C., Schiele, B., and Perona, P. (2012). Pedestrian detection: An evaluation of the state of the art. PAMI, 34.
  10. Evans, K. M., Jacobs, R. A., Tarduno, J. A., and Pelz, J. B. (2012). Collecting and analyzing eye-tracking data in outdoor environments. Journal of Eye Movement Research, pages 1-19.
  11. Felzenszwalb, P. F., Girshick, R. B., and McAllester, D. (2010). Cascade object detection with deformable part models. In CVPR, pages 2241-2248. IEEE.
  12. Fischler, M. A. and Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24(6):381-395.
  13. Gall, J. and Lempitsky, V. (2009). Class-specific hough forests for object detection. In CVPR.
  14. Henderson, J. M. (2003). Human gaze control during realworld scene perception. Trends in Cognitive Sciences, 7(11):498 - 504.
  15. Jokinen, K., Nishida, M., and Yamamoto, S. (2009). Eyegaze experiments for conversation monitoring. In Proc. of the 3rd IUCS, pages 303-308.
  16. Judd, T., Ehinger, K., Durand, F., and Torralba, A. (2009). Learning to predict where humans look. In ICCV, pages 2106-2113. IEEE.
  17. Leutenegger, S., Chli, M., and Siegwart, R. (2011). Brisk: Binary robust invariant scalable keypoints. In ICCV.
  18. Lowe, D. (2004). Distinctive image features from scaleinvariant keypoints. In IJCV, pages 91-110.
  19. Mathias, M., Benenson, R., Timofte, R., and Van Gool, L. (2013). Handling occlusions with franken-classifiers. Submitted for publication.
  20. Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., and Van Gool, L. (2005). A comparison of affine region detectors. IJCV, 65(1-2):43-72.
  21. Miksik, O. and Mikolajczyk, K. (2012). Evaluation of local detectors and descriptors for fast feature matching. In ICPR, pages 2681-2684.
  22. Rosten, E. and Drummond, T. (2005). Fusing points and lines for high performance tracking. In ICCV, ICCV, pages 1508-1515.
  23. Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011). Orb: an efficient alternative to sift or surf. In ICCV, pages 2564-2571.
  24. Toyama, T., Kieninger, T., Shafait, F., and Dengel, A. (2012). Gaze guided object recognition using a headmounted eye tracker. In ETRA, pages 91-98.
  25. Gompel, R. (2007). Eye Movements: A Window on Mind and Brain. Elsevier Science.
  26. Viola, P. and Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In CVPR, pages 511-518.
  27. Yun, K., Peng, Y., Samaras, D., Zelinsky, G. J., and Berg, T. L. (2013). Studying relationships between human gaze, description, and computer vision. In CVPR.
Download


Paper Citation


in Harvard Style

De Beugher S., Brône G. and Goedemé T. (2014). Automatic Analysis of In-the-Wild Mobile Eye-tracking Experiments using Object, Face and Person Detection . In Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2014) ISBN 978-989-758-003-1, pages 625-633. DOI: 10.5220/0004741606250633


in Bibtex Style

@conference{visapp14,
author={Stijn De Beugher and Geert Brône and Toon Goedemé},
title={Automatic Analysis of In-the-Wild Mobile Eye-tracking Experiments using Object, Face and Person Detection},
booktitle={Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2014)},
year={2014},
pages={625-633},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004741606250633},
isbn={978-989-758-003-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2014)
TI - Automatic Analysis of In-the-Wild Mobile Eye-tracking Experiments using Object, Face and Person Detection
SN - 978-989-758-003-1
AU - De Beugher S.
AU - Brône G.
AU - Goedemé T.
PY - 2014
SP - 625
EP - 633
DO - 10.5220/0004741606250633