Reinforcement Learning Guided by Provable Normative Compliance

Emery Neufeld

2022

Abstract

Reinforcement learning (RL) has shown promise as a tool for engineering safe, ethical, or legal behaviour in autonomous agents. Its use typically relies on assigning punishments to state-action pairs that constitute unsafe or unethical choices. Despite this assignment being a crucial step in this approach, however, there has been limited discussion on generalizing the process of selecting punishments and deciding where to apply them. In this paper, we adopt an approach that leverages an existing framework – the normative supervisor of (Neufeld et al., 2021) – during training. This normative supervisor is used to dynamically translate states and the applicable normative system into defeasible deontic logic theories, feed these theories to a theorem prover, and use the conclusions derived to decide whether or not to assign a punishment to the agent. We use multi- objective RL (MORL) to balance the ethical objective of avoiding violations with a non-ethical objective; we will demonstrate that our approach works for a multiplicity of MORL techniques, and show that it is effective regardless of the magnitude of the punishment we assign.

Paper Citation

in Harvard Style

Neufeld E. (2022). Reinforcement Learning Guided by Provable Normative Compliance. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART, ISBN 978-989-758-547-0, pages 444-453. DOI: 10.5220/0010835600003116

in Bibtex Style

@conference{icaart22,
author={Emery Neufeld},
title={Reinforcement Learning Guided by Provable Normative Compliance},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,},
year={2022},
pages={444-453},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010835600003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,
TI - Reinforcement Learning Guided by Provable Normative Compliance
SN - 978-989-758-547-0
AU - Neufeld E.
PY - 2022
SP - 444
EP - 453
DO - 10.5220/0010835600003116