Miniature Autonomous Vehicle Environment for Sim-to-Real Transfer in Reinforcement Learning

Stephan Pareigis, Daniel Riege, Tim Tiedemann

2024

Abstract

An experimental setup and preliminary validation of a platform for sim-to-real transfer in reinforcement learning for autonomous driving is presented. The platform features a 1:87 scale miniature autonomous vehicle, the tinycar, within a detailed miniature world that includes urban and rural settings. Key components include a simulation for training machine learning models, a digital twin with a tracking system using overhead cameras, an automatic repositioning mechanism of the miniature vehicle to reduce human intervention when training in the real-world, and an encoder based approach for reducing the state space dimension for the machine learning algorithms. The tinycar is equipped with a steering servo, DC motor, front-facing camera, and a custom PCB with an ESP32 micro-controller. A custom UDP-based network protocol enables real-time communication. The machine learning setup uses semantically segmented lanes of the streets as an input. These colored lanes can be directly produced by the simulation. In the real-world a machine learning based segmentation method is used to achieve the segmented lanes. Two methods are used to train a controller (actor): Imitation learning as a supervised learning method in which a Stanley controller serves as a teacher. Secondly, Twin Delayed Deep Deterministic Policy Gradient (TD3) is used to minimize the Cross-Track Error (CTE) of the miniature vehicle with respect to its lateral position in the street. Both methods are applied equally in simulation and in the real-world and are compared. Preliminary results show high accuracy in lane following and intersection navigation in simulation and real-world, supported by precise real-time feedback from the tracking system. While full integration of the RL model is ongoing, the presented results show the platform’s potential to further investigate the sim-to-real aspects in autonomous driving.

Paper Citation

in Harvard Style

Pareigis S., Riege D. and Tiedemann T. (2024). Miniature Autonomous Vehicle Environment for Sim-to-Real Transfer in Reinforcement Learning. In Proceedings of the 21st International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO; ISBN 978-989-758-717-7, SciTePress, pages 309-317. DOI: 10.5220/0012944400003822

in Bibtex Style

@conference{icinco24,
author={Stephan Pareigis and Daniel Riege and Tim Tiedemann},
title={Miniature Autonomous Vehicle Environment for Sim-to-Real Transfer in Reinforcement Learning},
booktitle={Proceedings of the 21st International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO},
year={2024},
pages={309-317},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012944400003822},
isbn={978-989-758-717-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 21st International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO
TI - Miniature Autonomous Vehicle Environment for Sim-to-Real Transfer in Reinforcement Learning
SN - 978-989-758-717-7
AU - Pareigis S.
AU - Riege D.
AU - Tiedemann T.
PY - 2024
SP - 309
EP - 317
DO - 10.5220/0012944400003822
PB - SciTePress