Good Practices on Hand Gestures Recognition for the Design of Customized NUI

Damiano Malafronte, Nicoletta Noceti

2015

Abstract

In this paper we consider the problem of recognizing dynamic human gestures in the context of human-machine interaction. We are particularly interested to the so-called Natural User Interfaces, a new modality based on a more natural and intuitive way of interacting with a digital device. In our work, a user can interact with a system by performing a set of encoded hand gestures in front of a webcam. We designed a method that first classifies hand poses guided by a finger detection procedure, and then recognizes known gestures with a syntactic approach. To this purpose, we collected a sequence of hand poses over time, to build a linguistic gesture description. The known gestures are formalized using a generative grammar. Then, at runtime, a parser allows us to perform gesture recognition leveraging on the production rules of the grammar. As for finger detection, we propose a new method which starts from a distance transform of the hand region and iteratively scans such region according to the distance values moving from a fingertip to the hand palm. We experimentally validated our approach, showing both the hand pose classification and gesture recognition performances.

References

  1. Bai, X. and Latecki, L. (2008). Path similarity skeleton graph matching. Trans. on PAMI, 30(7):1282-1292.
  2. Belongie, S., Malik, J., and Puzicha, J. (2002). Shape matching and object recognition using shape contexts. Trans. on PAMI, 24(4):509-522.
  3. Berg, M. d., Cheong, O., Kreveld, M. v., and Overmars, M. (2008). Computational Geometry: Algorithms and Applications.
  4. Bobick, A. F. and Wilson, A. D. (1997). A state-based approach to the representation and recognition of gesture. Trans. on PAMI, 19:1325-1337.
  5. Borgefors, G. (1986). Distance transformations in digital images. Comput. Vision Graph. Image Process., 34(3):344-371.
  6. Chen, Z., Kim, J., Liang, J., Zhang, J., and Yuan, Y. (2014). Real-Time Hand Gesture Recognition Using Finger Segmentation. The Scientific World Journal.
  7. Choi, J., Park, H., and Park, J. (2011). Hand shape recognition using distance transform and shape decomposition. In ICIP, pages 3605-3608.
  8. Chomsky, N. (1956). Three models for the description of language. Trans. on Information Theory, 2:113-124.
  9. Danielsson, P. (1980). Euclidean distance mapping. In Comp. Graph. and Image Proc., pages 227-248.
  10. Dardas, N. and Georganas, N. D. (2011). Realtime hand gesture detection and recognition using bag-of-features and support vector machine techniques. Trans. on Instrumentation and Measurement, 60(11):3592-3607.
  11. Dasarathy, B. V. (2002). Handbook of data mining and knowledge discovery. chapter Data Mining Tasks and Methods: Classification: Nearest-neighbor Approaches, pages 288-298.
  12. Droeschel, D., Stuckler, J., and Behnke, S. (2011). Learning to interpret pointing gestures with a time-of-flight camera. In Int. Conf. on HRI, pages 481-488.
  13. Fang, Y., Wang, K., Cheng, J., and Lu, H. (2007). A realtime hand gesture recognition method. In Int. Conf. on Multimedia and Expo, pages 995-998.
  14. Ghotkar, A. S. and Kharate, G. K. (2013). Vision based real time hand gesture recognition techniques for human computer interaction. Int. Jour. of Computer Applications, 70(16):1-8.
  15. Hu, M.-K. (1962). Visual pattern recognition by moment invariants. Trans. on Inf. Theory, 8(2):179-187.
  16. Ivanov, Y. A. and Bobick, A. F. (2000). Recognition of visual activities and interactions by stochastic parsing. Trans. on PAMI, 22(8):852-872.
  17. Joo, S.-W. and Chellappa, R. (2006). Attribute grammarbased event recognition and anomaly detection. In CVPRW, pages 107-107.
  18. Paulraj, Y. e. a. (2008). Extraction of head and hand gesture features for recognition of sign language. In Inter. Conf. on Electronic Design.
  19. Rautaray, S. and Agrawal, A. (2012). Vision based hand gesture recognition for human computer interaction: a survey. Artificial Intelligence Review, pages 1-54.
  20. Rauterberg, M. (1999). From gesture to action: Natural user interfaces.
  21. Ren, Z., Yuan, J., Meng, J., and Zhang, Z. (2013). Robust part-based hand gesture recognition using kinect sensor. Trans. on Multimedia, 15(5):1110-1120.
  22. Sato, Y., Kobayashi, Y., and Koike, H. (2000). Fast tracking of hands and fingertips in infrared images for augmented desk interface. In Int. Conf. on Automatic Face and Gesture Recognition, pages 462-467.
  23. Singh, S. K., Chauhan, D. S., Vatsa, M., and Singh, R. (2003). A robust skin color based face detection algorithm, tamkang. Jour. of Science and Engineering, 6:227-234.
  24. Viola, P. and Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In CVPR, volume 1, pages I-511-I-518.
  25. Zariffa, J. and Steeves, J. (2011). Computer vision-based classification of hand grip variations in neurorehabilitation. In ICORR, pages 1-4.
  26. Zivkovic, Z. (2004). Improved adaptive gaussian mixture model for background subtraction. In ICPR, volume 2, pages 28-31.
Download


Paper Citation


in Harvard Style

Malafronte D. and Noceti N. (2015). Good Practices on Hand Gestures Recognition for the Design of Customized NUI . In Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015) ISBN 978-989-758-090-1, pages 360-367. DOI: 10.5220/0005304203600367


in Bibtex Style

@conference{visapp15,
author={Damiano Malafronte and Nicoletta Noceti},
title={Good Practices on Hand Gestures Recognition for the Design of Customized NUI},
booktitle={Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015)},
year={2015},
pages={360-367},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005304203600367},
isbn={978-989-758-090-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015)
TI - Good Practices on Hand Gestures Recognition for the Design of Customized NUI
SN - 978-989-758-090-1
AU - Malafronte D.
AU - Noceti N.
PY - 2015
SP - 360
EP - 367
DO - 10.5220/0005304203600367