False Negative Reduction in Semantic Segmentation Under Domain Shift Using Depth Estimation

Kira Maag, Matthias Rottmann, Matthias Rottmann

2023

Abstract

State-of-the-Art deep neural networks demonstrate outstanding performance in semantic segmentation. However, their performance is tied to the domain represented by the training data. Open world scenarios cause inaccurate predictions which is hazardous in safety relevant applications like automated driving. In this work, we enhance semantic segmentation predictions using monocular depth estimation to improve segmentation by reducing the occurrence of non-detected objects in presence of domain shift. To this end, we infer a depth heatmap via a modified segmentation network which generates foreground-background masks, operating in parallel to a given semantic segmentation network. Both segmentation masks are aggregated with a focus on foreground classes (here road users) to reduce false negatives. To also reduce the occurrence of false positives, we apply a pruning based on uncertainty estimates. Our approach is modular in a sense that it post-processes the output of any semantic segmentation network. In our experiments, we observe less non-detected objects of most important classes and an enhanced generalization to other domains compared to the basic semantic segmentation prediction.

Paper Citation

in Harvard Style

Maag K. and Rottmann M. (2023). False Negative Reduction in Semantic Segmentation Under Domain Shift Using Depth Estimation. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 397-408. DOI: 10.5220/0011607400003417

in Bibtex Style

@conference{visapp23,
author={Kira Maag and Matthias Rottmann},
title={False Negative Reduction in Semantic Segmentation Under Domain Shift Using Depth Estimation},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP},
year={2023},
pages={397-408},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011607400003417},
isbn={978-989-758-634-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP
TI - False Negative Reduction in Semantic Segmentation Under Domain Shift Using Depth Estimation
SN - 978-989-758-634-7
AU - Maag K.
AU - Rottmann M.
PY - 2023
SP - 397
EP - 408
DO - 10.5220/0011607400003417
PB - SciTePress