A Supervised Learning Approach to Robust Reinforcement Learning for Job Shop Scheduling

Christoph Schmidl, Thiago Simão, Nils Jansen, Nils Jansen

2024

Abstract

The job shop scheduling problem (JSSP) is an NP-hard combinatorial optimization problem with the objective of minimizing the makespan while adhering to domain-specific constraints. Recent developments cast JSSP as a reinforcement learning (RL) problem, diverging from classical methods like heuristics or constraint programming. However, RL policies, serving as schedulers, often lack permutation invariance for job orderings in JSSP, limiting their generalization capabilities. In this paper, we improve the generalization of RL in the JSSP using a three-step approach that combines RL and supervised learning. Furthermore, we investigate permutation invariance and generalization to unseen JSSP instances. Initially, RL policies are trained on Taillard instances for 1800 seconds using Proximal Policy Optimization (PPO). These policies generate data sets of state-action pairs, augmented with varying permutation percentages to transpose job orders. The final step uses the generated data sets for retraining in a supervised learning setup, focusing on permutation invariance and dropout layers to improve robustness. Our approach (1) improves robustness regarding unseen instances by reducing the mean makespan and standard deviation after outlier removal by -0.43% and -15.31%, respectively, and (2) demonstrates the effect of job order permutations in supervised learning regarding the mean makespan and standard deviation.

Paper Citation

in Harvard Style

Schmidl C., Simão T. and Jansen N. (2024). A Supervised Learning Approach to Robust Reinforcement Learning for Job Shop Scheduling. In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART; ISBN 978-989-758-680-4, SciTePress, pages 1324-1335. DOI: 10.5220/0012473600003636

in Bibtex Style

@conference{icaart24,
author={Christoph Schmidl and Thiago Simão and Nils Jansen},
title={A Supervised Learning Approach to Robust Reinforcement Learning for Job Shop Scheduling},
booktitle={Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2024},
pages={1324-1335},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012473600003636},
isbn={978-989-758-680-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART
TI - A Supervised Learning Approach to Robust Reinforcement Learning for Job Shop Scheduling
SN - 978-989-758-680-4
AU - Schmidl C.
AU - Simão T.
AU - Jansen N.
PY - 2024
SP - 1324
EP - 1335
DO - 10.5220/0012473600003636
PB - SciTePress