SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completion

Juan Pablo Lagos, Esa Rahtu

2022

Abstract

Holistic scene understanding is pivotal for the performance of autonomous machines. In this paper we propose a new end-to-end model for performing semantic segmentation and depth completion jointly. The vast majority of recent approaches have developed semantic segmentation and depth completion as independent tasks. Our approach relies on RGB and sparse depth as inputs to our model and produces a dense depth map and the corresponding semantic segmentation image. It consists of a feature extractor, a depth completion branch, a semantic segmentation branch and a joint branch which further processes semantic and depth information altogether. The experiments done on Virtual KITTI 2 dataset, demonstrate and provide further evidence, that combining both tasks, semantic segmentation and depth completion, in a multi-task network can effectively improve the performance of each task. Code is available at https://github.com/juanb09111/semantic depth.

Paper Citation

in Harvard Style

Lagos J. and Rahtu E. (2022). SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completion. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 155-165. DOI: 10.5220/0010838500003124

in Bibtex Style

@conference{visapp22,
author={Juan Pablo Lagos and Esa Rahtu},
title={SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completion},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={155-165},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010838500003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completion
SN - 978-989-758-555-5
AU - Lagos J.
AU - Rahtu E.
PY - 2022
SP - 155
EP - 165
DO - 10.5220/0010838500003124
PB - SciTePress