Learned Fusion: 3D Object Detection Using Calibration-Free Transformer Feature Fusion

Michael Fürst, Michael Fürst, Rahul Jakkamsetty, René Schuster, René Schuster, Didier Stricker, Didier Stricker

2024

Abstract

The state of the art in 3D object detection using sensor fusion heavily relies on calibration quality, difficult to maintain in large scale deployment outside a lab environment. We present the first calibration-free approach for 3D object detection. Thus, eliminating complex and costly calibration procedures. Our approach uses transformers to map features between multiple views of different sensors at multiple abstraction levels. In an extensive evaluation for object detection, we show that our approach outperforms single modal setups by 14.1% in BEV mAP, and that the transformer learns mapping features. By showing calibration is not necessary for sensor fusion, we hope to motivate other researchers following the direction of calibration-free fusion. Additionally, resulting approaches have a substantial resilience against rotation and translation changes.

Paper Citation

in Harvard Style

Fürst M., Jakkamsetty R., Schuster R. and Stricker D. (2024). Learned Fusion: 3D Object Detection Using Calibration-Free Transformer Feature Fusion. In Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM; ISBN 978-989-758-684-2, SciTePress, pages 215-223. DOI: 10.5220/0012311400003654

in Bibtex Style

@conference{icpram24,
author={Michael Fürst and Rahul Jakkamsetty and René Schuster and Didier Stricker},
title={Learned Fusion: 3D Object Detection Using Calibration-Free Transformer Feature Fusion},
booktitle={Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM},
year={2024},
pages={215-223},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012311400003654},
isbn={978-989-758-684-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM
TI - Learned Fusion: 3D Object Detection Using Calibration-Free Transformer Feature Fusion
SN - 978-989-758-684-2
AU - Fürst M.
AU - Jakkamsetty R.
AU - Schuster R.
AU - Stricker D.
PY - 2024
SP - 215
EP - 223
DO - 10.5220/0012311400003654
PB - SciTePress