Optical Flow Augmented Semantic Segmentation Networks for Automated Driving

Hazem Rashed, Senthil Yogamani, Ahmad El-Sallab, Pavel Křížek, Mohamed El-Helw

2019

Abstract

Motion is a dominant cue in automated driving systems. Optical flow is typically computed to detect moving objects and to estimate depth using triangulation. In this paper, our motivation is to leverage the existing dense optical flow to improve the performance of semantic segmentation. To provide a systematic study, we construct four different architectures which use RGB only, flow only, RGBF concatenated and two-stream RGB + flow. We evaluate these networks on two automotive datasets namely Virtual KITTI and Cityscapes using the state-of-the-art flow estimator FlowNet v2. We also make use of the ground truth optical flow in Virtual KITTI to serve as an ideal estimator and a standard Farneback optical flow algorithm to study the effect of noise. Using the flow ground truth in Virtual KITTI, two-stream architecture achieves the best results with an improvement of 4% IoU. As expected, there is a large improvement for moving objects like trucks, vans and cars with 38%, 28% and 6% increase in IoU. FlowNet produces an improvement of 2.4% in average IoU with larger improvement in the moving objects corresponding to 26%, 11% and 5% in trucks, vans and cars. In Cityscapes, flow augmentation provided an improvement for moving objects like motorcycle and train with an increase of 17% and 7% in IoU.

Paper Citation

in Harvard Style

Rashed H., Yogamani S., El-Sallab A., Křížek P. and El-Helw M. (2019). Optical Flow Augmented Semantic Segmentation Networks for Automated Driving. In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 5: VISAPP; ISBN 978-989-758-354-4, SciTePress, pages 165-172. DOI: 10.5220/0007248301650172

in Bibtex Style

@conference{visapp19,
author={Hazem Rashed and Senthil Yogamani and Ahmad El-Sallab and Pavel Křížek and Mohamed El-Helw},
title={Optical Flow Augmented Semantic Segmentation Networks for Automated Driving},
booktitle={Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 5: VISAPP},
year={2019},
pages={165-172},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007248301650172},
isbn={978-989-758-354-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2019) - Volume 5: VISAPP
TI - Optical Flow Augmented Semantic Segmentation Networks for Automated Driving
SN - 978-989-758-354-4
AU - Rashed H.
AU - Yogamani S.
AU - El-Sallab A.
AU - Křížek P.
AU - El-Helw M.
PY - 2019
SP - 165
EP - 172
DO - 10.5220/0007248301650172
PB - SciTePress