Authors:
S. Ali Mirsoleimani
1
;
Aske Plaat
2
;
Jaap van den Herik
2
and
Jos Vermaseren
3
Affiliations:
1
Leiden University Niels Bohrweg 1 and Nikhef Theory Group, Netherlands
;
2
Leiden University Niels Bohrweg 1, Netherlands
;
3
Nikhef Theory Group, Netherlands
Keyword(s):
MCTS, Virtual Loss, Tree Parallelization, Search Overhead, Exploitation-exploration Trade-off.
Related
Ontology
Subjects/Areas/Topics:
Artificial Intelligence
;
Computational Intelligence
;
Evolutionary Computing
;
Formal Methods
;
Informatics in Control, Automation and Robotics
;
Intelligent Control Systems and Optimization
;
Knowledge Discovery and Information Retrieval
;
Knowledge-Based Systems
;
Machine Learning
;
Planning and Scheduling
;
Simulation and Modeling
;
Soft Computing
;
Symbolic Systems
Abstract:
Monte Carlo tree search algorithms, such as UCT, select the best-root-child as a result of an iterative search
process consistent with path dependency. Recent work has provided parallel methods that make the search
process faster. However, these methods violate the path-dependent nature of the sequential UCT process.
Here, a more rapid search thus results in a higher search overhead. The cost thereof is a lower time efficiency.
The concept of virtual loss is proposed to compensate for this cost. In this paper, we study the role of virtual
loss. Therefore, we conduct an empirical analysis of two methods for lock-free tree parallelization, viz. one
without virtual loss and one with the virtual loss. We use the UCT algorithm in the High Energy Physics
domain. In particular, we methodologically evaluate the performance of the both methods for a broad set
of configurations regarding search overhead and time efficiency. The results show that using virtual loss for
lock-free tree paralleliz
ation still degrades the performance of the algorithm. Contrary to what we aimed at.
(More)