Effect of Interaction Design of Reinforcement Learning Agents on Human Satisfaction in Partially Observable Domains

Divya Srivastava, Spencer Frazier, Mark Riedl, Karen M. Feigh

2021

Abstract

Interactive machine learning involves humans teaching with agents during their learning process. As this field grows, it is pertinent that laymen teachers, i.e. those without programming or extensive ML experience, are able to easily and effectively teach the agents. Previous work has investigated which factors contribute to the teacher’s experience when training agents in a fully observable domain. In this paper, we investigate how four different interaction methods affect agent performance and teacher experience in partially observable domains. As the domain in which the agent is learning becomes more complex, it accumulates less reward overall and needs more advice from the teacher. It is found that the most salient features that affect teacher satisfaction are agent compliance to advice, response speed, instruction quantity required, and reliability in agent response. It is suggested that machine learning algorithms incorporate a short time delay in the agent’s response and maximize the agent’s adherence to advice to increase reliability of the agent’s behavior. The need to generalize advice over time to reduce the amount of instruction needed varies depending on the presence of penalties in the environment.

Paper Citation

in Harvard Style

Srivastava D., Frazier S., Riedl M. and Feigh K. (2021). Effect of Interaction Design of Reinforcement Learning Agents on Human Satisfaction in Partially Observable Domains. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 2: HUCAPP; ISBN 978-989-758-488-6, SciTePress, pages 174-181. DOI: 10.5220/0010240101740181

in Bibtex Style

@conference{hucapp21,
author={Divya Srivastava and Spencer Frazier and Mark Riedl and Karen M. Feigh},
title={Effect of Interaction Design of Reinforcement Learning Agents on Human Satisfaction in Partially Observable Domains},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 2: HUCAPP},
year={2021},
pages={174-181},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010240101740181},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 2: HUCAPP
TI - Effect of Interaction Design of Reinforcement Learning Agents on Human Satisfaction in Partially Observable Domains
SN - 978-989-758-488-6
AU - Srivastava D.
AU - Frazier S.
AU - Riedl M.
AU - Feigh K.
PY - 2021
SP - 174
EP - 181
DO - 10.5220/0010240101740181
PB - SciTePress