Learning Based Interpretable End-to-End Control Using Camera Images

Sandesh Hiremath, Praveen Gummadi, Argtim Tika, Petrit Rama, Naim Bajcinca

2023

Abstract

This work proposes a learning-based controller for an autonomous vehicle to follow lanes on highways and motorways. The controller is designed as an interpretable deep neural network (DNN) that takes as input only a single image from the front-facing camera of an autonomous vehicle. To this end, we first implement an image-based model predictive controller (MPC) using a DNN, which takes as input 2D coordinates of the reference path made available as image pixels coordinates. Consequently, the DNN based controller can be seamlessly integrated with the perception and planner network to finally yield an end-to-end interpretable learning-based controller. Here, all of the controller components, namely- perception, planner, state estimation, and control synthesizer, are differentiable and thus capable of active and event-triggered adaptive training of the relevant components. The implemented network is tested in the CARLA simulation framework and then deployed in a real vehicle to finally demonstrate and validate its performance.

Paper Citation

in Harvard Style

Hiremath S., Gummadi P., Tika A., Rama P. and Bajcinca N. (2023). Learning Based Interpretable End-to-End Control Using Camera Images. In Proceedings of the 20th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO; ISBN 978-989-758-670-5, SciTePress, pages 474-484. DOI: 10.5220/0012206500003543

in Bibtex Style

@conference{icinco23,
author={Sandesh Hiremath and Praveen Gummadi and Argtim Tika and Petrit Rama and Naim Bajcinca},
title={Learning Based Interpretable End-to-End Control Using Camera Images},
booktitle={Proceedings of the 20th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO},
year={2023},
pages={474-484},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012206500003543},
isbn={978-989-758-670-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO
TI - Learning Based Interpretable End-to-End Control Using Camera Images
SN - 978-989-758-670-5
AU - Hiremath S.
AU - Gummadi P.
AU - Tika A.
AU - Rama P.
AU - Bajcinca N.
PY - 2023
SP - 474
EP - 484
DO - 10.5220/0012206500003543
PB - SciTePress