SAT-MARL: Specification Aware Training in Multi-Agent Reinforcement Learning

Fabian Ritz, Thomy Phan, Robert Müller, Thomas Gabor, Andreas Sedlmeier, Marc Zeller, Jan Wieghardt, Reiner Schmid, Horst Sauer, Cornel Klein, Claudia Linnhoff-Popien

2021

Abstract

A characteristic of reinforcement learning is the ability to develop unforeseen strategies when solving problems. While such strategies sometimes yield superior performance, they may also result in undesired or even dangerous behavior. In industrial scenarios, a system’s behavior also needs to be predictable and lie within defined ranges. To enable the agents to learn (how) to align with a given specification, this paper proposes to explicitly transfer functional and non-functional requirements into shaped rewards. Experiments are carried out on the smart factory, a multi-agent environment modeling an industrial lot-size-one production facility, with up to eight agents and different multi-agent reinforcement learning algorithms. Results indicate that compliance with functional and non-functional constraints can be achieved by the proposed approach.

Paper Citation

in Harvard Style

Ritz F., Phan T., Müller R., Gabor T., Sedlmeier A., Zeller M., Wieghardt J., Schmid R., Sauer H., Klein C. and Linnhoff-Popien C. (2021). SAT-MARL: Specification Aware Training in Multi-Agent Reinforcement Learning.In Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-484-8, pages 28-37. DOI: 10.5220/0010189500280037

in Bibtex Style

@conference{icaart21,
author={Fabian Ritz and Thomy Phan and Robert Müller and Thomas Gabor and Andreas Sedlmeier and Marc Zeller and Jan Wieghardt and Reiner Schmid and Horst Sauer and Cornel Klein and Claudia Linnhoff-Popien},
title={SAT-MARL: Specification Aware Training in Multi-Agent Reinforcement Learning},
booktitle={Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2021},
pages={28-37},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010189500280037},
isbn={978-989-758-484-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - SAT-MARL: Specification Aware Training in Multi-Agent Reinforcement Learning
SN - 978-989-758-484-8
AU - Ritz F.
AU - Phan T.
AU - Müller R.
AU - Gabor T.
AU - Sedlmeier A.
AU - Zeller M.
AU - Wieghardt J.
AU - Schmid R.
AU - Sauer H.
AU - Klein C.
AU - Linnhoff-Popien C.
PY - 2021
SP - 28
EP - 37
DO - 10.5220/0010189500280037