Balancing Exploitation and Exploration via Fully Probabilistic Design of Decision Policies

Miroslav Kárný, František Hůla

2019

Abstract

Adaptive decision making learns an environment model serving a design of a decision policy. The policy-generated actions influence both the acquired reward and the future knowledge. The optimal policy properly balances exploitation with exploration. The inherent dimensionality curse of decision making under incomplete knowledge prevents the realisation of the optimal design. This has stimulated repetitive attempts to reach this balance at least approximately. Usually, either: (a) the exploitative reward is enriched by a part reflecting the exploration quality and a feasible approximate certainty-equivalent design is made; or (b) an explorative random noise is added to the purely exploitative actions. This paper avoids the inauspicious (a) and improves (b) by employing the non-standard fully probabilistic design (FPD) of decision policies, which naturally generates random actions. Monte-Carlo experiments confirm its achieved quality. The quality stems from methodological contributions, which include: (i) an improvement of the relation between FPD and standard Markov decision processes; (ii) a design of an adaptive tuning of an FPD-parameter. The latter also suits for the tuning of the temperature in both simulated annealing and Boltzmann’s machine.

Paper Citation

in Harvard Style

Kárný M. and Hůla F. (2019). Balancing Exploitation and Exploration via Fully Probabilistic Design of Decision Policies.In Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-350-6, pages 857-864. DOI: 10.5220/0007587208570864

in Bibtex Style

@conference{icaart19,
author={Miroslav Kárný and František Hůla},
title={Balancing Exploitation and Exploration via Fully Probabilistic Design of Decision Policies},
booktitle={Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2019},
pages={857-864},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007587208570864},
isbn={978-989-758-350-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Balancing Exploitation and Exploration via Fully Probabilistic Design of Decision Policies
SN - 978-989-758-350-6
AU - Kárný M.
AU - Hůla F.
PY - 2019
SP - 857
EP - 864
DO - 10.5220/0007587208570864