Authors:
Michael Fürst
1
;
2
;
Rahul Jakkamsetty
2
;
René Schuster
1
;
2
and
Didier Stricker
1
;
2
Affiliations:
1
Augmented Vision, RPTU, University of Kaiserslautern-Landau, Kaiserslautern, Germany
;
2
DFKI, German Research Center for Artificial Intelligence, Kaiserslautern, Germany
Keyword(s):
3D Object Detection, Calibration Free, Sensor Fusion, Transformer, Self-Attention.
Abstract:
The state of the art in 3D object detection using sensor fusion heavily relies on calibration quality, difficult to maintain in large scale deployment outside a lab environment. We present the first calibration-free approach for 3D object detection. Thus, eliminating complex and costly calibration procedures. Our approach uses transformers to map features between multiple views of different sensors at multiple abstraction levels. In an extensive evaluation for object detection, we show that our approach outperforms single modal setups by 14.1% in BEV mAP, and that the transformer learns mapping features. By showing calibration is not necessary for sensor fusion, we hope to motivate other researchers following the direction of calibration-free fusion. Additionally, resulting approaches have a substantial resilience against rotation and translation changes.