Fiducial Points-supported Object Pose Tracking on RGB Images via Particle Filtering with Heuristic Optimization

Mateusz Majcher, Bogdan Kwolek

2021

Abstract

We present an algorithm for tracking 6D pose of the object in a sequence of RGB images. The images are acquired by a calibrated camera. A particle filter is utilized to estimate the posterior probability distribution of the object poses. The probabilistic observation model is built on the projected 3D model onto image and then matching the rendered object with the segmented object. It is determined using object silhouette and distance transform-based edge scores. A hypothesis about 6D object pose that is calculated on the basis of object keypoints and the PnP algorithm is included in the probability distribution. A k-means++ algorithm is then executed on multi-modal probability distribution to determine modes. A multi-swarm particle swarm optimization is executed afterwards to find finest modes in the probability distribution together with the best pose. The object of interest is segmented by an U-Net neural network. Eight fiducial points of the object are determined by a neural network. A data generator employing 3D object models has been developed to synthesize photorealistic images with ground-truth data for training neural networks both for object segmentation and estimation of keypoints. The 6D object pose tracker has been evaluated both on synthetic and real images. We demonstrate experimentally that object pose hypotheses calculated on the basis of fiducial points and the PnP algorithm lead to considerable improvements in tracking accuracy.

Paper Citation

in Harvard Style

Majcher M. and Kwolek B. (2021). Fiducial Points-supported Object Pose Tracking on RGB Images via Particle Filtering with Heuristic Optimization. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP; ISBN 978-989-758-488-6, SciTePress, pages 919-926. DOI: 10.5220/0010237109190926

in Bibtex Style

@conference{visapp21,
author={Mateusz Majcher and Bogdan Kwolek},
title={Fiducial Points-supported Object Pose Tracking on RGB Images via Particle Filtering with Heuristic Optimization},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP},
year={2021},
pages={919-926},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010237109190926},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP
TI - Fiducial Points-supported Object Pose Tracking on RGB Images via Particle Filtering with Heuristic Optimization
SN - 978-989-758-488-6
AU - Majcher M.
AU - Kwolek B.
PY - 2021
SP - 919
EP - 926
DO - 10.5220/0010237109190926
PB - SciTePress