Pose Guided Feature Learning for 3D Object Tracking on RGB Videos

Mateusz Majcher, Bogdan Kwolek

2022

Abstract

In this work we propose a new approach to 3D object pose tracking in sequences of RGB images acquired by a calibrated camera. A single hourglass neural network that has been trained to detect fiducial keypoints on a set of objects delivers heatmaps representing 2D locations of the keypoints. Given a calibrated camera model and a sparse object model consisting of 3D locations of the keypoints, the keypoints in hypothesized object poses are projected onto 2D plane and then matched with the heatmaps. A quaternion particle filter with a probabilistic observation model that uses such a matching is employed to maintain 3D object pose distribution. A single Siamese neural network is trained for a set of objects on keypoints from the current and previous frame in order to generate a particle in the predicted 3D object pose. The filter draws particles to predict the current pose using its a priori knowledge about the object velocity and includes the predicted 3D object pose by the neural network in a priori distribution. Thus, the hypothesized 3D object poses are generated using both a priori knowledge about the object velocity in 3D and keypoint-based geometric reasoning as well as relative transformations in the image plane. In an extended algorithm we combine the set of propagated particles with an optimized particle, whose pose is determined by Levenberg-Marguardt.

Paper Citation

in Harvard Style

Majcher M. and Kwolek B. (2022). Pose Guided Feature Learning for 3D Object Tracking on RGB Videos. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 574-581. DOI: 10.5220/0010886800003124

in Bibtex Style

@conference{visapp22,
author={Mateusz Majcher and Bogdan Kwolek},
title={Pose Guided Feature Learning for 3D Object Tracking on RGB Videos},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={574-581},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010886800003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - Pose Guided Feature Learning for 3D Object Tracking on RGB Videos
SN - 978-989-758-555-5
AU - Majcher M.
AU - Kwolek B.
PY - 2022
SP - 574
EP - 581
DO - 10.5220/0010886800003124
PB - SciTePress