A Depth-based Approach for 3D Dynamic Gesture Recognition

Hajar Hiyadi, Fakhreddine Ababsa, Christophe Montagne, El Houssine Bouyakhf, Fakhita Regragui

2015

Abstract

In this paper we propose a recognition technique of 3D dynamic gesture for human robot interaction (HRI) based on depth information provided by Kinect sensor. The body is tracked using the skeleton algorithm provided by the Kinect SDK. The main idea of this work is to compute the angles of the upper body joints which are active when executing gesture. The variation of these angles are used as inputs of Hidden Markov Models (HMM) in order to recognize the dynamic gestures. Results demonstrate the robustness of our method against environmental conditions such as illumination changes and scene complexity due to using depth information only.

References

  1. Barczak, A. and Dadgostar, F. (2005). Real-time hand tracking using a set of cooperative classifiers based on haarlike features. In Research Letters in the Information and Mathematical Sciences.
  2. Bengio, Y. and Frasconi, P. (1996). Ieee transactions on neural networks. In Input-output HMMs for sequence processing.
  3. Binh, N. D. and Ejima, T. (2002). Real-time hand gesture recognition using pseudo 3-d hidden markov model. In Proceedings of the 5th IEEE International Conference on Cognitive Informatics (ICCI 7806).
  4. Bleiweiss, A. and Werman, M. (2009). Fusion time-of- ight depth and color for realtime segmentation and tracking. In DAGM Symposium for Pattern Recognition.
  5. Boesen, A., Larsen, L., Hauberg, S., , and Pedersen, K. S. (2011). Unscented kalman filtering for articulated human tracking. In 17th Scandinavian Conference, SCIA.
  6. Chen, Q., Georganas, N., and Petriu, E. (2008). Hand gesture recognition using haar-like features and a stochastic context-free grammar. In IEEE Transactions on Instrumentation and Measurement.
  7. Corradini, A. (2001). Dynamic time warping for off-line recognition of a small gesture vocabulary. In ICCV Workshop on RecognitionAnalysis, and Tracking of Faces and Gestures in Real-Time Systems.
  8. Eickeler, S., Kosmala, A., and Rigoll, G. (1998). Hidden markov model based continuous online gesture recognition. In Proceedings of 14th International Conference on Pattern Recognition.
  9. Elmezain, M., Al-Hamadi, A., Appenrodt, J., and Michaelis, B. (2009). A hidden markov model-based isolated and meaningful hand gesture recognition. In Journal of WSCG.
  10. Elmezain, M., Al-Hamadi, A., and B.Michaelis (2008). Real-time capable system for handgesture recognition using hidden markov models in stereo color image sequences. In Journal of WSCG.
  11. F., A. (2009). Robust extended kalman filtering for camera pose tracking using 2d to 3d lines correspondences. In International Conference on Advanced Intelligent Mechatronics.
  12. F., A. and Mallem, M. (2006). Robust line tracking using a particle filter for camera pose estimation. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology.
  13. G., Y., D., H., O., Y., and Weihua, S. (2012). Human gesture recognition through a kinect sensor. In Robotics and Biomimetics (ROBIO).
  14. Ghobadi, S., Leopprich, O., Hartmann, K., and Loffeld, O. (2007). Hand segmentation using 2d/3d images. In Proceeding of image and Vision Computiong.
  15. Gu, Y., Do, H., and Sheng, Y. O. W. (2012). Human gesture recognition through a kinect sensor. In International Conference on Robotics and Biomimetics.
  16. Koller, D., Thrun, S., PlagemannVarun, C., and Ganapathi, V. (2010). Real time identification and localization of body parts from depth images. In IEEE International Conference on Robotics and Automation (ICRA).
  17. Lawrence, R. (1989). A tutorial on hidden markov models and selected applications in speech recognition. In Proceeding of the IEEE.
  18. Li, M., Cattani, C., and Chen, S. (2011). Viewing sea level by a one-dimensional random function with long memory. In Mathematical Problems in Engineering.
  19. Matuszek, C., Bo, L., Zettlemoyer, L., and Fox, D. (2014). Learning from unscripted deictic gesture and language for human-robot interactions. In I. J. Robotic.
  20. Qin, S., Zhu, X., Yang, Y., and Jiang, Y. (2014). Realtime hand gesture recognition from depth images using convex shape decomposition method. In Journal of Signal Processing Systems.
  21. Rautaray, S. S. and Agrawal, A. (2011). A real time hand tracking system for interactive applications. In International journal of computer Applications.
  22. Saon, G. and Chien, J. T. (2012). Bayesian sensing hidden markov models. In IEEE Transactions on Audio, Speech, and Language Processing.
  23. Wang, S., Quattoni, A., Morency, L., Demirdjian, D., and Darrell, T. (2006). Hidden conditional random fields for gesture recognition. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR).
  24. Wang, X., Xia, M., Cai, H., Gao, Y., and Cattani, C. (2012). Hidden-markov-models-based dynamic hand gesture recognition. In Mathematical Problems in Engineering.
  25. Xia, L., Chen, C.-C., and Aggarwal, J. (2011). Human detection using depth information by kinect. In Computer Society Conference on Computer Vision and Pattern Recognition - CVPR.
  26. Xu, D., Chen, Y.-L., Wu, X., and Xu, Y. (2011). Integrated approach of skincolor detection and depth information for hand and face localization. In IEEE International Conference on Robotics and Biomimetics - ROBIO.
  27. Xu, D., Wu, X., Chen, Y., and Xu, Y. (2014). Online dynamic gesture recognition for human robot interaction. In IEEE Journal of Intelligent and Robotic Systems.
  28. Xu, J., Wu, Y., and Katsaggelos, A. (2010). Part-based initialization for hand tracking. In The 17th IEEE International Conference on Image Processing (ICIP).
  29. Zhang, Z. (2012). Microsoft kinect sensor and its effect. In Multi Media.
Download


Paper Citation


in Harvard Style

Hiyadi H., Ababsa F., Montagne C., Bouyakhf E. and Regragui F. (2015). A Depth-based Approach for 3D Dynamic Gesture Recognition . In Proceedings of the 12th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO, ISBN 978-989-758-123-6, pages 103-110. DOI: 10.5220/0005545401030110


in Bibtex Style

@conference{icinco15,
author={Hajar Hiyadi and Fakhreddine Ababsa and Christophe Montagne and El Houssine Bouyakhf and Fakhita Regragui},
title={A Depth-based Approach for 3D Dynamic Gesture Recognition},
booktitle={Proceedings of the 12th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,},
year={2015},
pages={103-110},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005545401030110},
isbn={978-989-758-123-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,
TI - A Depth-based Approach for 3D Dynamic Gesture Recognition
SN - 978-989-758-123-6
AU - Hiyadi H.
AU - Ababsa F.
AU - Montagne C.
AU - Bouyakhf E.
AU - Regragui F.
PY - 2015
SP - 103
EP - 110
DO - 10.5220/0005545401030110