Grouping of Maintenance Actions with Deep Reinforcement Learning and Graph Convolutional Networks

David Kerkkamp, Zaharah Bukhsh, Yingqian Zhang, Nils Jansen

2022

Abstract

Reinforcement learning (RL) has shown promising performance in several applications such as robotics and games. However, the use of RL in emerging real-world domains such as smart industry and asset management remains scarce. This paper addresses the problem of optimal maintenance planning using historical data. We propose a novel Deep RL (DRL) framework based on Graph Convolutional Networks (GCN) to leverage the inherent graph structure of typical assets. As demonstrator, we employ an underground sewer pipe network. In particular, instead of dispersed maintenance actions of individual pipes across the network, the GCN ensures the grouping of maintenance actions of geographically close pipes. We perform experiments using the distinct physical characteristics, deterioration profiles, and historical data of sewer inspections within an urban environment. The results show that combining Deep Q-Networks (DQN) with GCN leads to structurally more reliable networks and a higher degree of maintenance grouping, compared to DQN with fully-connected layers and standard preventive and corrective maintenance strategy that are often adopted in practice. Our approach shows potential for developing efficient and practical maintenance plans in terms of cost and reliability.

Paper Citation

in Harvard Style

Kerkkamp D., Bukhsh Z., Zhang Y. and Jansen N. (2022). Grouping of Maintenance Actions with Deep Reinforcement Learning and Graph Convolutional Networks. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-547-0, pages 574-585. DOI: 10.5220/0010907500003116

in Bibtex Style

@conference{icaart22,
author={David Kerkkamp and Zaharah Bukhsh and Yingqian Zhang and Nils Jansen},
title={Grouping of Maintenance Actions with Deep Reinforcement Learning and Graph Convolutional Networks},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2022},
pages={574-585},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010907500003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Grouping of Maintenance Actions with Deep Reinforcement Learning and Graph Convolutional Networks
SN - 978-989-758-547-0
AU - Kerkkamp D.
AU - Bukhsh Z.
AU - Zhang Y.
AU - Jansen N.
PY - 2022
SP - 574
EP - 585
DO - 10.5220/0010907500003116