Self-scaling Kinematic Hand Skeleton for Real-time 3D Hand-finger Pose Estimation

Kristian Ehlers, Jan Helge Klüssendorff

2015

Abstract

Since low cost RGB-D sensors have been available, gesture detection has gained more and more interest in the field of human computer and human robot interaction. It is possible to navigate through interactive menus by waving the hand and to confirm menu items by pointing at them. Such applications require real-time body or hand-finger pose estimation algorithms. This paper presents a kinematic approach to estimate the full pose of the hand including the finger joints’ angles. A self-scaling kinematic hand skeleton model is presented and fitted into the 3D data of the hand in real-time on standard hardware with up to 30 frames per second without using a GPU. This approach is based on least-square minimization and an intelligent choice of the error function. The tracking accuracy is evaluated based on a recorded dataset as well as simulated data. Qualitative results are presented emphasizing the tracking ability under hard conditions like full hand turning and self-occlusion.

References

  1. Aristidou, A. and Lasenby, J. (2010). Motion Capture with Constrained Inverse Kinematics for Real-Time Hand Tracking. In International Symposium on Communications, Control and Signal Processing, number March, pages 3-5.
  2. Athitsos, V. and Sclaroff, S. (2003). Estimating 3D hand pose from a cluttered image. 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2003 Proceedings, 2:II-432-9.
  3. Ballan, L., Taneja, A., Gall, J., Gool, L. V., and Pollefeys, M. (2012). Motion Capture of Hands in Action Using Discriminative Salient Points. In Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., and Schmid, C., editors, Computer Vision - ECCV 2012, volume 7577 of Lecture Notes in Computer Science, pages 640-653.
  4. ElKoura, G. and Singh, K. (2003). Handrix: animating the human hand. Eurographics symposium on Computer animation.
  5. Erol, A., Bebis, G., Nicolescu, M., Boyle, R. D., and Twombly, X. (2007). Vision-based hand pose estimation: A review. Computer Vision and Image Understanding, 108(1-2):52-73.
  6. Gorce, M. D. L., Fleet, D. J., and Paragios, N. (2011). Model-Based 3D Hand Pose Estimation from Monocular Video. In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 33, pages 1793- 1805. Laboratoire MAS, Ecole Centrale de Paris, Chatenay-Malabry, IEEE.
  7. Han, J., Shao, L., Xu, D., and Shotton, J. (2013). Enhanced computer vision with Microsoft Kinect sensor: a review. IEEE transactions on cybernetics, 43(5):1318- 34.
  8. Horaud, R., Forbes, F., Yguel, M., Dewaele, G., and Zhang, J. (2011). Rigid and articulated point registration with expectation conditional maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(3):587-602.
  9. Keskin, C., , Kirac, F., Kara, Y. E., and Akarun, L. (2011). Real time hand pose estimation using depth sensors. In Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on, pages 1228- 1234.
  10. Keskin, C., , Kirac, F., Kara, Y. E., and Akarun, L. (2012). Hand Pose Estimation and Hand Shape Classification Using Multi-layered Randomized Decision Forests, volume 7577. Springer Berlin Heidelberg.
  11. Lee, J. and Kunii, T. (1995). Model-based analysis of hand posture. Computer Graphics and Applications, IEEE, 15(5):77-86.
  12. Liang, H., Yuan, J., and Thalmann, D. (2012). Hand Pose Estimation by Combining Fingertip Tracking and Articulated ICP. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry, VRCAI 7812, pages 87-90, New York, NY, USA. ACM.
  13. Oikonomidis, I., Kyriazis, N., and Argyros, A. (2011). Efficient model-based 3D tracking of hand articulations using Kinect. Procedings of the British Machine Vision Conference, pages 101.1-101.11.
  14. Oikonomidis, I., Kyriazis, N., and Argyros, A. A. (2010). Markerless and Efficient 26-DOF Hand Pose Recovery. Hand The, pages 744-757.
  15. Qian, C., Sun, X., Wei, Y., Tang, X., and Sun, J. (2014). Realtime and Robust Hand Tracking from Depth. In IEEComputer Vision and Pattern Recognition.
  16. Raheja, J. L., Chaudhary, A., and Singal, K. (2011). Tracking of Fingertips and Centers of Palm Using KINECT. Third International Conference on Computational Intelligence Modelling Simulation, pages 248-252.
  17. Ren, Z., Meng, J., and Yuan, J. (2011). Depth Camera Based Hand Gesture Recognition and its Applications in Human-Computer-Interaction. IEEE International Conference on Information Communication and Signal Processing, (1):3-7.
  18. Ren, Z. and Yuan, J. (2011). Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera. Proceedings of the 19th ACM international, pages 1-4.
  19. Schröder, M., Elbrechter, C., Maycock, J., Haschke, R., Botsch, M., and Ritter, H. (2012). Real-Time Hand Tracking with a Color Glove for the Actuation of Anthropomorphic Robot Hands. In Proceedings of IEEERAS International Conference on Humanoid Robots, pages 262-269.
  20. Schröder, M., Maycock, J., Ritter, H., and Botsch, M. (2013). Analysis of Hand Synergies for Inverse Kinematics Hand Tracking. In IEEE International Conference on Robotics and Automation, Workshop of ”Hand synergies - how to tame the complexity of grasping”.
  21. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. (2011). Real-time human pose recognition in parts from single depth images. IEEE Conference on Computer Vision and Pattern Recognition, pages 1297-1304.
  22. Sridhar, S., Oulasvirta, A., and Theobalt, C. (2013). Interactive Markerless Articulated Hand Motion Tracking Using RGB and Depth Data. 2013 IEEE International Conference on Computer Vision, pages 2456-2463.
  23. Wang, R., Paris, S., and Popovic, J. (2011). 6D Hands: Markerless Hand-tracking for Computer Aided Design. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST 7811, pages 549-558, New York, NY, USA. ACM.
  24. Wang, R. Y. and Popovic, J. (2009). Real-time handtracking with a color glove. ACM Transactions on Graphics, 28(3):1.
  25. Wang, Y., Min, J., Zhang, J., Liu, Y., Xu, F., Dai, Q., and Chai, J. (2013). Video-based hand manipulation capture through composite motion control. ACM Transactions on Graphics, 32(4):1.
  26. Zhao, W., Zhang, J., Min, J., and Chai, J. (2013). Robust realtime physics-based motion control for human grasping. ACM Transactions on Graphics, 32(6):1-12.
Download


Paper Citation


in Harvard Style

Ehlers K. and Klüssendorff J. (2015). Self-scaling Kinematic Hand Skeleton for Real-time 3D Hand-finger Pose Estimation . In Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015) ISBN 978-989-758-090-1, pages 185-196. DOI: 10.5220/0005257501850196


in Bibtex Style

@conference{visapp15,
author={Kristian Ehlers and Jan Helge Klüssendorff},
title={Self-scaling Kinematic Hand Skeleton for Real-time 3D Hand-finger Pose Estimation},
booktitle={Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015)},
year={2015},
pages={185-196},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005257501850196},
isbn={978-989-758-090-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015)
TI - Self-scaling Kinematic Hand Skeleton for Real-time 3D Hand-finger Pose Estimation
SN - 978-989-758-090-1
AU - Ehlers K.
AU - Klüssendorff J.
PY - 2015
SP - 185
EP - 196
DO - 10.5220/0005257501850196