Hybrid Feature based Pyramid Network for Nighttime Semantic Segmentation

Yuqi Li, Yinan Ma, Jing Wu, Chengnian Long

2021

Abstract

In recent years, considerable progress has been made on semantic segmentation tasks. However, most existing works focus on only day-time images under favorable illumination conditions. In this work, we aim at nighttime semantic segmentation, which is remaining to be solved due to the problems of over-and under-exposures caused by complex lighting conditions and the lack of trainable nighttime dataset as pixel-level annotation requires extensive time and human effort. We (1) propose a hybrid network combining image pyramid network and Gray Level Co-occurrence Matrix (GLCM). GLCM is a significant descriptor of texture information, as statistical features to compensate the missing texture information in the over-and under-exposures problem at night. (2) design an exposure-awareness encoder network by fusing hybrid features hierarchically in GLCM fusion layers. (3) elaborately generate a trainable nighttime dataset, Carla-based Synthesis Nighttime dataset (CSN dataset), with 10027 synthesis images to resolve the problem of large-scale human annotations. To check whether the network trained on synthesized images is effective in the real world we also collect a real-world dataset called NightCampus with 500 nighttime images with annotations used as test dataset. We prove that our network trained on synthetic dataset yielding top performances on our real-world dataset.

Paper Citation

in Harvard Style

Li Y., Ma Y., Wu J. and Long C. (2021). Hybrid Feature based Pyramid Network for Nighttime Semantic Segmentation. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP; ISBN 978-989-758-488-6, SciTePress, pages 321-328. DOI: 10.5220/0010248503210328

in Bibtex Style

@conference{visapp21,
author={Yuqi Li and Yinan Ma and Jing Wu and Chengnian Long},
title={Hybrid Feature based Pyramid Network for Nighttime Semantic Segmentation},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP},
year={2021},
pages={321-328},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010248503210328},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 4: VISAPP
TI - Hybrid Feature based Pyramid Network for Nighttime Semantic Segmentation
SN - 978-989-758-488-6
AU - Li Y.
AU - Ma Y.
AU - Wu J.
AU - Long C.
PY - 2021
SP - 321
EP - 328
DO - 10.5220/0010248503210328
PB - SciTePress