Learning to Correct Reconstructions from Multiple Views

Ștefan Săftescu, Paul Newman

2021

Abstract

This paper is about reducing the cost of building good large-scale reconstructions post-hoc. This is an important consideration for survey vehicles which are equipped with sensors which offer mixed fidelity or are restricted by road rules to high-speed traversals. We render 2D views of an existing, lower-quality, reconstruction and train a convolutional neural network (CNN) that refines inverse-depth to match to a higher-quality reconstruction. Since the views that we correct are rendered from the same reconstruction, they share the same geometry, so overlapping views complement each other. We impose a loss during training which guides predictions on neighbouring views to have the same geometry and has been shown to improve performance. In contrast to previous work, which corrects each view independently, we also make predictions on sets of neighbouring views jointly. This is achieved by warping feature maps between views and thus bypassing memory-intensive computation. We make the observation that features in the feature maps are viewpoint-dependent, and propose a method for transforming features with dynamic filters generated by a multi-layer perceptron from the relative poses between views. In our experiments we show that this last step is necessary for successfully fusing feature maps between views.

Paper Citation

in Harvard Style

Săftescu Ș. and Newman P. (2021). Learning to Correct Reconstructions from Multiple Views. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP; ISBN 978-989-758-488-6, SciTePress, pages 901-909. DOI: 10.5220/0010226409010909

in Bibtex Style

@conference{visapp21,
author={Ștefan Săftescu and Paul Newman},
title={Learning to Correct Reconstructions from Multiple Views},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP},
year={2021},
pages={901-909},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010226409010909},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP
TI - Learning to Correct Reconstructions from Multiple Views
SN - 978-989-758-488-6
AU - Săftescu Ș.
AU - Newman P.
PY - 2021
SP - 901
EP - 909
DO - 10.5220/0010226409010909
PB - SciTePress