Global Reward Design for Cooperative Agents to Achieve Flexible Production Control under Real-time Constraints

Sebastian Pol, Schirin Baer, Danielle Turner, Vladimir Samsonov, Tobias Meisen

2021

Abstract

In flexible manufacturing, efficient production requires reactive control. We present a solution for solving practical and flexible job shop scheduling problems, focusing on minimizing total makespan while dealing with many product variants and unseen production scenarios. In our system, each product is controlled by an independent reinforcement learning agent for resource allocation and transportation. A significant challenge in multi-agent solutions is collaboration between agents for a common optimization objective. We implement and compare two global reward designs enabling cooperation between the agents during production. Namely, we use dense local rewards augmented with global reward factors, and a sparse global reward design. The agents are trained on randomized product combinations. We validate the results using unseen scheduling scenarios to evaluate generalization. Our goal is not to outperform existing domain-specific heuristics for total makespan, but to generate comparably good schedules with the advantage of being able to instantaneously react to unforeseen events. While the implemented reward designs show very promising results, the dense reward design performs slightly better while the sparse reward design is much more intuitive to implement. We benchmark our results against simulated annealing based on total makespan and computation time, showing that we achieve comparable results with reactive behavior.

Paper Citation

in Harvard Style

Pol S., Baer S., Turner D., Samsonov V. and Meisen T. (2021). Global Reward Design for Cooperative Agents to Achieve Flexible Production Control under Real-time Constraints. In Proceedings of the 23rd International Conference on Enterprise Information Systems - Volume 1: ICEIS, ISBN 978-989-758-509-8, pages 515-526. DOI: 10.5220/0010455805150526

in Bibtex Style

@conference{iceis21,
author={Sebastian Pol and Schirin Baer and Danielle Turner and Vladimir Samsonov and Tobias Meisen},
title={Global Reward Design for Cooperative Agents to Achieve Flexible Production Control under Real-time Constraints},
booktitle={Proceedings of the 23rd International Conference on Enterprise Information Systems - Volume 1: ICEIS,},
year={2021},
pages={515-526},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010455805150526},
isbn={978-989-758-509-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 23rd International Conference on Enterprise Information Systems - Volume 1: ICEIS,
TI - Global Reward Design for Cooperative Agents to Achieve Flexible Production Control under Real-time Constraints
SN - 978-989-758-509-8
AU - Pol S.
AU - Baer S.
AU - Turner D.
AU - Samsonov V.
AU - Meisen T.
PY - 2021
SP - 515
EP - 526
DO - 10.5220/0010455805150526