Memory-Based Learning of Global Control Policies from Local Controllers

Daniel Nikovski, Junmin Zhong, Junmin Zhong, William Yerazunis

2024

Abstract

The paper proposes a novel method for constructing a global control policy, valid everywhere in the state space of a dynamical system, from a set of solutions computed for specific initial states in that space by means of differential dynamic programming. The global controller chooses controls based on elements of the pre-computed solutions, leveraging the property that these solutions compute not only nominal state and control trajectories from the initial states, but also a set of linear controllers that can stabilize the system around the nominal trajectories, as well as a set of localized estimators of the optimal cost-to-go for system states around the nominal states. An empirical verification of three variants of the algorithm on two benchmark problems demonstrates that making use of the cost-to-go estimators results in the best performance (lowest average cost) and often leads to dramatic reduction in the number of pre-computed solutions that have to be stored in memory, which in its turn speeds up control computation in real time.

Paper Citation

in Harvard Style

Nikovski D., Zhong J. and Yerazunis W. (2024). Memory-Based Learning of Global Control Policies from Local Controllers. In Proceedings of the 21st International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO; ISBN 978-989-758-717-7, SciTePress, pages 237-244. DOI: 10.5220/0012921900003822

in Bibtex Style

@conference{icinco24,
author={Daniel Nikovski and Junmin Zhong and William Yerazunis},
title={Memory-Based Learning of Global Control Policies from Local Controllers},
booktitle={Proceedings of the 21st International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO},
year={2024},
pages={237-244},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012921900003822},
isbn={978-989-758-717-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 21st International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO
TI - Memory-Based Learning of Global Control Policies from Local Controllers
SN - 978-989-758-717-7
AU - Nikovski D.
AU - Zhong J.
AU - Yerazunis W.
PY - 2024
SP - 237
EP - 244
DO - 10.5220/0012921900003822
PB - SciTePress