Let’s Do the Time Warp Again: Human Action Assistance for Reinforcement Learning Agents

Carter Burn, Frederick Crabbe, Rebecca Hwa

2021

Abstract

Reinforcement learning (RL) agents may take a long time to learn a policy for a complex task. One way to help the agent to convergence on a policy faster is by offering it some form of assistance from a teacher who already has some expertise on the same task. The teacher can be either a human or another computer agent, and they can provide assistance by controlling the reward, action selection, or state definition that the agent views. However, some forms of assistance might come more naturally from a human teacher than a computer teacher and vice versa. For instance, a challenge for human teachers in providing action selection is that because computers and human operate at different speed increments, it is difficult to translate what constitutes an action selection for a particular state in a human’s perception to that of the computer agent. In this paper, we introduce a system called Time Warp that allows a human teacher to provide action selection assistance to the agent during critical moments of the training for the RL agent. We find that Time Warp is able to help the agent develop a better policy in less time than an RL agent with no assistance and rivals the performance of computer teaching agents. Time Warp also is able to reach the results with only ten minutes of human training time.

Paper Citation

in Harvard Style

Burn C., Crabbe F. and Hwa R. (2021). Let’s Do the Time Warp Again: Human Action Assistance for Reinforcement Learning Agents.In Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-484-8, pages 92-100. DOI: 10.5220/0010258700920100

in Bibtex Style

@conference{icaart21,
author={Carter Burn and Frederick Crabbe and Rebecca Hwa},
title={Let’s Do the Time Warp Again: Human Action Assistance for Reinforcement Learning Agents},
booktitle={Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2021},
pages={92-100},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010258700920100},
isbn={978-989-758-484-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - Let’s Do the Time Warp Again: Human Action Assistance for Reinforcement Learning Agents
SN - 978-989-758-484-8
AU - Burn C.
AU - Crabbe F.
AU - Hwa R.
PY - 2021
SP - 92
EP - 100
DO - 10.5220/0010258700920100