Search for Robust Policies in Reinforcement Learning

Qi Li

2020

Abstract

While Reinforcement Learning (RL) often operates in idealized Markov Decision Processes (MDPs), their applications in real-world tasks often encounter noise such as in uncertain initial state distributions, noisy dynamics models. Further noise can also be introduced in actions, rewards, and the observations. In this paper we specifically focus on the problem of making agents act in a robust manner under different observation noise distributions for during training and for during testing. Such characterization of training and testing distributions is not common in RL as it is more common to train and deploy the agent on the same MDP. In this work, two methods of improving agent robustness to observation noise - training on noisy environments and modifying the reward function directly to encourage stable policies, are proposed and evaluated. We show that by training on noisy observation distributions, even if the distribution is different from the one in test, can benefit agent performance in test, while the reward modifications are less generally applicable, only improving the optimisation in some cases.

Paper Citation

in Harvard Style

Li Q. (2020). Search for Robust Policies in Reinforcement Learning. In Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-395-7, pages 421-428. DOI: 10.5220/0008917404210428

in Bibtex Style

@conference{icaart20,
author={Qi Li},
title={Search for Robust Policies in Reinforcement Learning},
booktitle={Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2020},
pages={421-428},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008917404210428},
isbn={978-989-758-395-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Search for Robust Policies in Reinforcement Learning
SN - 978-989-758-395-7
AU - Li Q.
PY - 2020
SP - 421
EP - 428
DO - 10.5220/0008917404210428