Trajectory Generation Model: Building a Simulation Link Between Expert Knowledge and Offline Learning

Arlena Wellßow, Arlena Wellßow, Torben Logemann, Eric MSP Veith, Eric MSP Veith

2024

Abstract

Reinforcement learning has shown its worth in multiple fields (e. g., voltage control, market participation). However, training each agent from scratch leads to relatively long time and high computational power costs. Including expert knowledge in agents is beneficial, as human reasoning is coming up with independent solutions for unusual states of the (less complex) system, and, especially in long-known fields, many strategies are already established and, therefore, learnable for the agent. Using this knowledge allows agents to use these solutions without encountering such situations in the first place. Expert knowledge is usually available only in semi-structured, non-machine-readable forms, such as (mis-) use cases. Also, data containing extreme situations, grid states, or emergencies is usually limited. However, these situations can be described as scenarios in these semi-structured forms. A State machine representing the scenarios’ steps can be built from thereon to generate data, which can then be used for offline learning. This paper shows a model and a prototype for such state machines. We implemented this prototype using state machines as internal policies of agents without a learning part for four specified scenarios. Acting according to the actuator setpoints the state machines emit, the agent outputs control the connected simulation. We found that our implemented prototype can generate data in the context of smart grid simulation. These data samples show the specified behavior, cover the search space through variance, and do not include data that violates given constraints.

Paper Citation

in Harvard Style

Wellßow A., Logemann T. and MSP Veith E. (2024). Trajectory Generation Model: Building a Simulation Link Between Expert Knowledge and Offline Learning. In Proceedings of the 14th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH; ISBN 978-989-758-708-5, SciTePress, pages 91-102. DOI: 10.5220/0012764900003758

in Bibtex Style

@conference{simultech24,
author={Arlena Wellßow and Torben Logemann and Eric MSP Veith},
title={Trajectory Generation Model: Building a Simulation Link Between Expert Knowledge and Offline Learning},
booktitle={Proceedings of the 14th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH},
year={2024},
pages={91-102},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012764900003758},
isbn={978-989-758-708-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH
TI - Trajectory Generation Model: Building a Simulation Link Between Expert Knowledge and Offline Learning
SN - 978-989-758-708-5
AU - Wellßow A.
AU - Logemann T.
AU - MSP Veith E.
PY - 2024
SP - 91
EP - 102
DO - 10.5220/0012764900003758
PB - SciTePress