A Bare-Hand Gesture Interaction System for Virtual Environments

Benjamín Hernández, Alejandro Flores

2014

Abstract

Hand-based gestures provide direct mappings of user actions to 3D UI tasks; they are becoming a more attractive interaction alternative than keyboards, mice, controllers, among others. In this paper, we present a fast algorithm for hand gesture recognition for interaction in virtual environments. The method applies mathematical morphology operators (binarization and dilation) to acquire a clean segmented hand image from a depth data stream on which a curvature-metric and K-means algorithm is applied to detect the fingertips, then using fingertip and palm positions together with anthropomorphic metrics and a rule based system we perform gesture recognition. In addition, the intermittence gesture spotting problem is reduced using a digital integrator. Finally, a set of virtual environments were designed to demonstrate the performance, reliability and feasibility of our method.

References

  1. Argyros, A. A. and Lourakis, M. I. A. (2006). Vision-based interpretation of hand gestures for remote control of a computer mouse. In Proceedings of the 2006 international conference on Computer Vision in HumanComputer Interaction. Springer-Verlag.
  2. DeFanti, T. and Sandin, D. (1977). Final report to the national endowment of the arts. In US NEA r60-34-163.
  3. Frati, V. and Prattichizzo, D. (2011). Using kinect for hand tracking and rendering in wearable haptics. In World Haptics Conference (WHC), 2011 IEEE. IEEE Computer Society Press.
  4. Garg, P., Aggarwal, N., and Sofat, S. (2009). Vision based hand gesture recognition. In 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing. IEEE Computer Society Press.
  5. Greiner, T. (1991). Hand Anthropometry of U.S. Army Personnel. Technical report (U.S. Army Natic Laboratories. Development Engineering Center). U.S. Army Natick Research, Development & Engineering Center.
  6. Iason Oikonomidis, N. K. and Argyros, A. (2011). Efficient model-based 3d tracking of hand articulations using kinect. In Proceedings of the British Machine Vision Conference. BMVA Press.
  7. Iddan, G. J. and Yahav, G. (2001). G.: 3d imaging in the studio (and elsewhere). In: SPIE.
  8. Liang, H., Yuan, J., and Thalmann, D. (2012). 3d fingertip and palm tracking in depth image sequences. In Proceedings of the 20th ACM international conference on Multimedia. ACM.
  9. Liu, X. and Fujimura, K. (2004). Hand gesture recognition using depth data. In Proceedings of the Sixth IEEE international conference on Automatic face and gesture recognition. IEEE Computer Society Press.
  10. Oikonomidis, I., Kyriazis, N., and Argyros, A. A. (2011). Full dof tracking of a hand interacting with an object by modeling occlusions and physical constraints. In Proceedings of the 2011 International Conference on Computer Vision. IEEE Computer Society Press.
  11. Plamondon, R. and Srihari, S. N. (2000). On-line and offline handwriting recognition: A comprehensive survey. In IEEE Transactions on Pattern Analysis and Machine Intelligence. IEEE Computer Society Press.
  12. Raheja, J., Chaudhary, A., and Singal, K. (2011). Tracking of fingertips and centers of palm using kinect. In Computational Intelligence, Modelling and Simulation (CIMSiM), 2011 Third International Conference on.
  13. Rehg, J. M. and Kanade, T. (1993). Digiteyes: Vision-based human hand tracking. Technical report, School of Computer Science, Carnegie Mellon University, 1993.
  14. Ren, Z., Yuan, J., and Zhang, Z. (2011). Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera. In Proceedings of the 19th ACM international conference on Multimedia. ACM.
  15. Rubner, Y., Tomasi, C., and Guibas, L. J. (2000). The earth movers distance as a metric for image retrieval. In International Journal of Computer Vision. Srpinger.
  16. Sturman, D. J. and Zeltzer, D. (1994). A survey of glovebased input. In IEEE Comput. Graph. Appl. IEEE Computer Society Press.
  17. Suau, X., Casas, J. R., and Ruiz-Hidalgo, J. (2011). Realtime head and hand tracking. In Multimedia and Expo (ICME), 2011 IEEE International Conference on. IEEE Computer Society Press.
  18. Utsumi, A. (1997). Direct manipulation scene creation in 3d: estimating hand postures from multiple-camera images. In Proceedings of ACM SIGGRAPH 97. ACM.
  19. Wang, R., Paris, S., and Popovic, J. (2011). 6d hands: markerless hand-tracking for computer aided design. In Proceedings of the 24th annual ACM symposium on User interface software and technology. ACM.
  20. Wobbrock, J. O., Wilson, A. D., and Li, Y. (2007). Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In Proceedings of the 20th annual ACM symposium on User interface software and technology. ACM.
  21. Zabulis, X., Baltzakis, H., and Argyros, A. A. (2009). Vision-based hand gesture recognition for human computer interaction. In The Universal Access Handbook. Lawrence Erlbaum Associates, Inc. (LEA), Series on ”Human Factors and Ergonomics”.
  22. Zimmerman, T. G., Lanier, J., Blanchard, C., Bryson, S., and Harvill, Y. (1987). A hand gesture interface device. In Proceedings of the SIGCHI/GI. ACM.
Download


Paper Citation


in Harvard Style

Hernández B. and Flores A. (2014). A Bare-Hand Gesture Interaction System for Virtual Environments . In Proceedings of the 9th International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2014) ISBN 978-989-758-002-4, pages 464-471. DOI: 10.5220/0004695204640471


in Bibtex Style

@conference{grapp14,
author={Benjamín Hernández and Alejandro Flores},
title={A Bare-Hand Gesture Interaction System for Virtual Environments},
booktitle={Proceedings of the 9th International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2014)},
year={2014},
pages={464-471},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004695204640471},
isbn={978-989-758-002-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, (VISIGRAPP 2014)
TI - A Bare-Hand Gesture Interaction System for Virtual Environments
SN - 978-989-758-002-4
AU - Hernández B.
AU - Flores A.
PY - 2014
SP - 464
EP - 471
DO - 10.5220/0004695204640471