Integrated Data-Driven Framework for Automatic Controller Tuning with Setpoint Stabilization Through Reinforcement Learning

Babak Mohajer, Neelaksh Singh, Joram Liebeskind

2024

Abstract

We introduce a three-stage framework for designing an optimal controller. First, we apply offline black-box optimization algorithms to find optimal controller parameters based on a heuristically chosen setpoint profile and a novel cost function for penalizing control signal oscillations and direction changes. Then, we leverage cloud data to generate device-specific setpoint profiles and tune the controller parameters to perform well on the device with respect to the same cost function. Finally, we train a control policy on top of the offline tuned controller after deployment on device through an online learning algorithm to handle unseen setpoint variations. A novel reward function encouraging setpoint stabilization is added for preventing destabilization from coupling effects. Bayesian Optimization and Nelder-Mead methods are used for offline optimization, and a state-of-the-art model free Reinforcement Learning algorithm namely Soft Actor-Critic is used for online optimization. We validate our framework using a realistic HVAC hydraulic circuit simulation.

Paper Citation

in Harvard Style

Mohajer B., Singh N. and Liebeskind J. (2024). Integrated Data-Driven Framework for Automatic Controller Tuning with Setpoint Stabilization Through Reinforcement Learning. In Proceedings of the 14th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH; ISBN 978-989-758-708-5, SciTePress, pages 431-442. DOI: 10.5220/0012861100003758

in Bibtex Style

@conference{simultech24,
author={Babak Mohajer and Neelaksh Singh and Joram Liebeskind},
title={Integrated Data-Driven Framework for Automatic Controller Tuning with Setpoint Stabilization Through Reinforcement Learning},
booktitle={Proceedings of the 14th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH},
year={2024},
pages={431-442},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012861100003758},
isbn={978-989-758-708-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH
TI - Integrated Data-Driven Framework for Automatic Controller Tuning with Setpoint Stabilization Through Reinforcement Learning
SN - 978-989-758-708-5
AU - Mohajer B.
AU - Singh N.
AU - Liebeskind J.
PY - 2024
SP - 431
EP - 442
DO - 10.5220/0012861100003758
PB - SciTePress