Outdoor Scenes Pixel-wise Semantic Segmentation using Polarimetry and Fully Convolutional Network

Marc Blanchon, Olivier Morel, Yifei Zhang, Ralph Seulin, Nathan Crombez, Désiré Sidibé

2019

Abstract

In this paper, we propose a novel method for pixel-wise scene segmentation application using polarimetry. To address the difficulty of detecting highly reflective areas such as water and windows, we use the angle and degree of polarization of these areas, obtained by processing images from a polarimetric camera. A deep learning framework, based on encoder-decoder architecture, is used for the segmentation of regions of interest. Different methods of augmentation have been developed to obtain a sufficient amount of data, while preserving the physical properties of the polarimetric images. Moreover, we introduce a new dataset comprising both RGB and polarimetric images with manual ground truth annotations for seven different classes. Experimental results on this dataset, show that deep learning can benefit from polarimetry and obtain better segmentation results compared to RGB modality. In particular, we obtain an improvement of 38.35% and 22.92% in the accuracy for segmenting windows and cars respectively.

Paper Citation

in Harvard Style

Blanchon M., Morel O., Zhang Y., Seulin R., Crombez N. and Sidibé D. (2019). Outdoor Scenes Pixel-wise Semantic Segmentation using Polarimetry and Fully Convolutional Network. In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 5: VISAPP; ISBN 978-989-758-354-4, SciTePress, pages 328-335. DOI: 10.5220/0007360203280335

in Bibtex Style

@conference{visapp19,
author={Marc Blanchon and Olivier Morel and Yifei Zhang and Ralph Seulin and Nathan Crombez and Désiré Sidibé},
title={Outdoor Scenes Pixel-wise Semantic Segmentation using Polarimetry and Fully Convolutional Network},
booktitle={Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 5: VISAPP},
year={2019},
pages={328-335},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007360203280335},
isbn={978-989-758-354-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 5: VISAPP
TI - Outdoor Scenes Pixel-wise Semantic Segmentation using Polarimetry and Fully Convolutional Network
SN - 978-989-758-354-4
AU - Blanchon M.
AU - Morel O.
AU - Zhang Y.
AU - Seulin R.
AU - Crombez N.
AU - Sidibé D.
PY - 2019
SP - 328
EP - 335
DO - 10.5220/0007360203280335
PB - SciTePress