Multi-stream CNN based Video Semantic Segmentation for Automated Driving

Ganesh Sistu, Sumanth Chennupati, Senthil Yogamani

Abstract

Majority of semantic segmentation algorithms operate on a single frame even in the case of videos. In this work, the goal is to exploit temporal information within the algorithm model for leveraging motion cues and temporal consistency. We propose two simple high-level architectures based on Recurrent FCN (RFCN) and Multi-Stream FCN (MSFCN) networks. In case of RFCN, a recurrent network namely LSTM is inserted between the encoder and decoder. MSFCN combines the encoders of different frames into a fused encoder via 1x1 channel-wise convolution. We use a ResNet50 network as the baseline encoder and construct three networks namely MSFCN of order 2 & 3 and RFCN of order 2. MSFCN-3 produces the best results with an accuracy improvement of 9% and 15% for Highway and New York-like city scenarios in the SYNTHIACVPR’ 16 dataset using mean IoU metric. MSFCN-3 also produced 11% and 6% for SegTrack V2 and DAVIS datasets over the baseline FCN network. We also designed an efficient version of MSFCN-2 and RFCN-2 using weight sharing among the two encoders. The efficient MSFCN-2 provided an improvement of 11% and 5% for KITTI and SYNTHIA with negligible increase in computational complexity compared to the baseline version.

Download


Paper Citation


in Harvard Style

Sistu G., Chennupati S. and Yogamani S. (2019). Multi-stream CNN based Video Semantic Segmentation for Automated Driving.In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, ISBN 978-989-758-354-4, pages 173-180. DOI: 10.5220/0007248401730180


in Bibtex Style

@conference{visapp19,
author={Ganesh Sistu and Sumanth Chennupati and Senthil Yogamani},
title={Multi-stream CNN based Video Semantic Segmentation for Automated Driving},
booktitle={Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP,},
year={2019},
pages={173-180},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007248401730180},
isbn={978-989-758-354-4},
}


in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP,
TI - Multi-stream CNN based Video Semantic Segmentation for Automated Driving
SN - 978-989-758-354-4
AU - Sistu G.
AU - Chennupati S.
AU - Yogamani S.
PY - 2019
SP - 173
EP - 180
DO - 10.5220/0007248401730180