Reinforcement Learning with Quantitative Verification for Assured Multi-Agent Policies

Joshua Riley, Radu Calinescu, Colin Paterson, Daniel Kudenko, Alec Banks

2021

Abstract

In multi-agent reinforcement learning, several agents converge together towards optimal policies that solve complex decision-making problems. This convergence process is inherently stochastic, meaning that its use in safety-critical domains can be problematic. To address this issue, we introduce a new approach that combines multi-agent reinforcement learning with a formal verification technique termed quantitative verification. Our assured multi-agent reinforcement learning approach constrains agent behaviours in ways that ensure the satisfaction of requirements associated with the safety, reliability, and other non-functional aspects of the decision-making problem being solved. The approach comprises three stages. First, it models the problem as an abstract Markov decision process, allowing quantitative verification to be applied. Next, this abstract model is used to synthesise a policy which satisfies safety, reliability, and performance constraints. Finally, the synthesised policy is used to constrain agent behaviour within the low-level problem with a greatly lowered risk of constraint violations. We demonstrate our approach using a safety-critical multi-agent patrolling problem.

Paper Citation

in Harvard Style

Riley J., Calinescu R., Paterson C., Kudenko D. and Banks A. (2021). Reinforcement Learning with Quantitative Verification for Assured Multi-Agent Policies.In Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-484-8, pages 237-245. DOI: 10.5220/0010258102370245

in Bibtex Style

@conference{icaart21,
author={Joshua Riley and Radu Calinescu and Colin Paterson and Daniel Kudenko and Alec Banks},
title={Reinforcement Learning with Quantitative Verification for Assured Multi-Agent Policies},
booktitle={Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2021},
pages={237-245},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010258102370245},
isbn={978-989-758-484-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Reinforcement Learning with Quantitative Verification for Assured Multi-Agent Policies
SN - 978-989-758-484-8
AU - Riley J.
AU - Calinescu R.
AU - Paterson C.
AU - Kudenko D.
AU - Banks A.
PY - 2021
SP - 237
EP - 245
DO - 10.5220/0010258102370245