Incorporating Temporal Information into 3D Hand Pose Estimation Using Scene Flow

Niklas Hermes, Niklas Hermes, Alexander Bigalke, Mattias Heinrich

2024

Abstract

In this paper we present a novel approach that uses 3D point cloud sequences to integrate temporal information and spatial constraints into existing 3D hand pose estimation methods in order to establish an improved prediction of 3D hand poses. We utilize scene flow to match correspondences between two point sets and present a method that optimizes and harnesses existing scene flow networks for the application of 3D hand pose estimation. For increased generalizability, we propose a module that learns to recognize spatial hand pose associations to transform existing poses into a low-dimensional pose space. In a comprehensive evaluation on the public dataset NYU, we show the benefits of our individual modules and provide insights into the generalization capabilities and the behaviour of our method with noisy data. Furthermore, we demonstrate that our method reduces the error of existing state-of-the-art 3D hand pose estimation methods by up to 7.6%. With a speed of over 40 fps our method is real-time capable and can be integrated into existing 3D hand pose estimation methods with little computational overhead.

Paper Citation

in Harvard Style

Hermes N., Bigalke A. and Heinrich M. (2024). Incorporating Temporal Information into 3D Hand Pose Estimation Using Scene Flow. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 286-294. DOI: 10.5220/0012361400003660

in Bibtex Style

@conference{visapp24,
author={Niklas Hermes and Alexander Bigalke and Mattias Heinrich},
title={Incorporating Temporal Information into 3D Hand Pose Estimation Using Scene Flow},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP},
year={2024},
pages={286-294},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012361400003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP
TI - Incorporating Temporal Information into 3D Hand Pose Estimation Using Scene Flow
SN - 978-989-758-679-8
AU - Hermes N.
AU - Bigalke A.
AU - Heinrich M.
PY - 2024
SP - 286
EP - 294
DO - 10.5220/0012361400003660
PB - SciTePress