Multi-Task Planar Reconstruction with Feature Warping Guidance

Luan Wei, Anna Hilsmann, Peter Eisert, Peter Eisert

2024

Abstract

Piece-wise planar 3D reconstruction simultaneously segments plane instances and recovers their 3D plane parameters from an image, which is particularly useful for indoor or man-made environments. Efficient reconstruction of 3D planes coupled with semantic predictions offers advantages for a wide range of applications requiring scene understanding and concurrent spatial mapping. However, most existing planar reconstruction models either neglect semantic predictions or do not run efficiently enough for real-time applications. We introduce SOLOPlanes, a real-time planar reconstruction model based on a modified instance segmentation architecture which simultaneously predicts semantics for each plane instance, along with plane parameters and piece-wise plane instance masks. We achieve an improvement in instance mask segmentation by including multi-view guidance for plane predictions in the training process. This cross-task improvement, training for plane prediction but improving the mask segmentation, is due to the nature of feature sharing in multi-task learning. Our model simultaneously predicts semantics using single images at inference time, while achieving real-time predictions at 43 FPS. Code is available at: https://github.com/fraunhoferhhi/SOLOPlanes.

Paper Citation

in Harvard Style

Wei L., Hilsmann A. and Eisert P. (2024). Multi-Task Planar Reconstruction with Feature Warping Guidance. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 652-661. DOI: 10.5220/0012396200003660

in Bibtex Style

@conference{visapp24,
author={Luan Wei and Anna Hilsmann and Peter Eisert},
title={Multi-Task Planar Reconstruction with Feature Warping Guidance},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2024},
pages={652-661},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012396200003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - Multi-Task Planar Reconstruction with Feature Warping Guidance
SN - 978-989-758-679-8
AU - Wei L.
AU - Hilsmann A.
AU - Eisert P.
PY - 2024
SP - 652
EP - 661
DO - 10.5220/0012396200003660
PB - SciTePress