An Analysis of Virtual Loss in Parallel MCTS

S. Ali Mirsoleimani, Aske Plaat, Jaap van den Herik, Jos Vermaseren

2017

Abstract

Monte Carlo tree search algorithms, such as UCT, select the best-root-child as a result of an iterative search process consistent with path dependency. Recent work has provided parallel methods that make the search process faster. However, these methods violate the path-dependent nature of the sequential UCT process. Here, a more rapid search thus results in a higher search overhead. The cost thereof is a lower time efficiency. The concept of virtual loss is proposed to compensate for this cost. In this paper, we study the role of virtual loss. Therefore, we conduct an empirical analysis of two methods for lock-free tree parallelization, viz. one without virtual loss and one with the virtual loss. We use the UCT algorithm in the High Energy Physics domain. In particular, we methodologically evaluate the performance of the both methods for a broad set of configurations regarding search overhead and time efficiency. The results show that using virtual loss for lock-free tree parallelization still degrades the performance of the algorithm. Contrary to what we aimed at.

References

  1. Chaslot, G., Winands, M., and van den Herik, J. (2008a). Parallel Monte-Carlo Tree Search. In the 6th Internatioal Conference on Computers and Games, volume 5131, pages 60-71. Springer Berlin Heidelberg.
  2. Chaslot, G. M. J. B., Winands, M. H. M., van den Herik, J., Uiterwijk, J. W. H. M., and Bouzy, B. (2008b). Progressive strategies for Monte-Carlo tree search. New Mathematics and Natural Computation, 4(03):343- 357.
  3. Coulom, R. (2006). Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. In Proceedings of the 5th International Conference on Computers and Games, volume 4630 of CG'06, pages 72-83. Springer-Verlag.
  4. Enzenberger, M. and Müller, M. (2010). A lock-free multithreaded Monte-Carlo tree search algorithm. Advances in Computer Games, 6048:14-20.
  5. Gelly, S. and Silver, D. (2007). Combining online and offline knowledge in UCT. In the 24th International Conference on Machine Learning, pages 273-280, New York, USA. ACM Press.
  6. Kocsis, L. and Szepesvári, C. (2006). Bandit based MonteCarlo Planning Levente. In Fürnkranz, J., Scheffer, T., and Spiliopoulou, M., editors, ECML'06 Proceedings of the 17th European conference on Machine Learning, volume 4212 of Lecture Notes in Computer Science, pages 282-293. Springer Berlin Heidelberg.
  7. Kuipers, J., Plaat, A., Vermaseren, J., and van den Herik, J. (2013). Improving Multivariate Horner Schemes with Monte Carlo Tree Search. Computer Physics Communications, 184(11):2391-2395.
  8. Ruijl, B., Vermaseren, J., Plaat, A., and van den Herik, J. (2014). Combining Simulated Annealing and Monte Carlo Tree Search for Expression Simplification. Proceedings of ICAART Conference 2014, 1(1):724-731.
  9. Sephton, N., Cowling, P. I., Powley, E., Whitehouse, D., and Slaven, N. H. (2014). Parallelization of Information Set Monte Carlo Tree Search. In Evolutionary Computation (CEC), 2014 IEEE Congress on, pages 2290-2297.
  10. Soejima, Y., Kishimoto, A., and Watanabe, O. (2010). Evaluating Root Parallelization in Go. IEEE Transactions on Computational Intelligence and AI in Games, 2(4):278-287.
  11. Teytaud, F. and Dehos, J. (2015). One the Tactical and Strategic Behaviour of MCTS When Biasing Random Simulations. ICCA Journal, 38(2):67-80.
Download


Paper Citation


in Harvard Style

Mirsoleimani S., Plaat A., van den Herik J. and Vermaseren J. (2017). An Analysis of Virtual Loss in Parallel MCTS . In Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-220-2, pages 648-652. DOI: 10.5220/0006205806480652


in Bibtex Style

@conference{icaart17,
author={S. Ali Mirsoleimani and Aske Plaat and Jaap van den Herik and Jos Vermaseren},
title={An Analysis of Virtual Loss in Parallel MCTS},
booktitle={Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2017},
pages={648-652},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006205806480652},
isbn={978-989-758-220-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - An Analysis of Virtual Loss in Parallel MCTS
SN - 978-989-758-220-2
AU - Mirsoleimani S.
AU - Plaat A.
AU - van den Herik J.
AU - Vermaseren J.
PY - 2017
SP - 648
EP - 652
DO - 10.5220/0006205806480652