A New Approach Combining Trained Single-view Networks with Multi-view Constraints for Robust Multi-view Object Detection and Labelling

Yue Zhang, Adrian Hilton, Jean-Yves Guillemaut

2020

Abstract

We propose a multi-view framework for joint object detection and labelling based on pairs of images. The proposed framework extends the single-view Mask R-CNN approach to multiple views without need for additional training. Dedicated components are embedded into the framework to match objects across views by enforcing epipolar constraints, appearance feature similarity and class coherence. The multi-view extension enables the proposed framework to detect objects which would otherwise be mis-detected in a classical Mask R-CNN approach, and achieves coherent object labelling across views. By avoiding the need for additional training, the approach effectively overcomes the current shortage of multi-view datasets. The proposed framework achieves high quality results on a range of complex scenes, being able to output class, bounding box, mask and an additional label enforcing coherence across views. In the evaluation, we show qualitative and quantitative results on several challenging outdoor multi-view datasets and perform a comprehensive comparison to verify the advantages of the proposed method.

Paper Citation

in Harvard Style

Zhang Y., Hilton A. and Guillemaut J. (2020). A New Approach Combining Trained Single-view Networks with Multi-view Constraints for Robust Multi-view Object Detection and Labelling. In Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 5: VISAPP; ISBN 978-989-758-402-2, SciTePress, pages 452-461. DOI: 10.5220/0008991104520461

in Bibtex Style

@conference{visapp20,
author={Yue Zhang and Adrian Hilton and Jean-Yves Guillemaut},
title={A New Approach Combining Trained Single-view Networks with Multi-view Constraints for Robust Multi-view Object Detection and Labelling},
booktitle={Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 5: VISAPP},
year={2020},
pages={452-461},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008991104520461},
isbn={978-989-758-402-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 5: VISAPP
TI - A New Approach Combining Trained Single-view Networks with Multi-view Constraints for Robust Multi-view Object Detection and Labelling
SN - 978-989-758-402-2
AU - Zhang Y.
AU - Hilton A.
AU - Guillemaut J.
PY - 2020
SP - 452
EP - 461
DO - 10.5220/0008991104520461
PB - SciTePress