Monte Carlo Tree Search in The Octagon Theory

Hugo Fernandes, Pedro Nogueira, Eugénio Oliveira

2014

Abstract

Monte Carlo Tree Search (MCTS) is a family of algorithms known by its performance in difficult problems that cannot be targeted with the current technology using classical AI approaches. This paper discusses the application of MCTS techniques in the fixed-length game The Octagon Theory, comparing various policies and enhancements with the best known greedy approach and standard Monte Carlo Search. The experiments reveal that the usage of Move Groups, Decisive Moves, Upper Confidence Bounds for Trees (UCT) and Limited Simulation Lengths turn a losing MCTS agent into the best performing one in a domain with estimated gametree complexity of 10293, even when the provided computational budget is kept low.

References

  1. Allis, V. L. (1994). Searching for Solutions in Games and Artificial Intelligence. PhD thesis, University of Limburg.
  2. Browne, C., Powley, E., Whitehouse, D., Lucas, S., Cowling, P., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., and Colton, S. (2012). A survey of monte carlo tree search methods. Computational Intelligence and AI in Games, IEEE Transactions on, 4(1):1-43.
  3. Cazenave, T. and Helmstetter, B. (2005). Combining tactical search and monte-carlo in the game of go. In IN: CIG05, pages 171-175.
  4. Chaslot, G., Bakkes, S., Szita, I., and Spronck, P. (2008). Monte-carlo tree search: A new framework for game ai. In Darken, C. and Mateas, M., editors, AIIDE. The AAAI Press.
  5. Childs, B., Brodeur, J., and Kocsis, L. (2008). Transpositions and move groups in monte carlo tree search. In Computational Intelligence and Games, 2008. CIG 7808. IEEE Symposium On, pages 389 -395.
  6. Coulom, R. (2007). Efficient selectivity and backup operators in monte-carlo tree search. In Proceedings of the 5th international conference on Computers and games, CG'06, pages 72-83, Berlin, Heidelberg.
  7. Den Teuling, N. (2011). Monte-carlo tree search for the simultaneous move game tron. Univ. Maastricht, Netherlands, Tech. Rep.
  8. Kocsis, L., Szepesvári, C., and Willemson, J. (2006). Improved monte-carlo search. Univ. Tartu, Estonia, Tech. Rep, 1.
  9. Kroeker, K. L. (2011). A new benchmark for artificial intelligence. Commun. ACM, 54(8):13-15.
  10. Müller, M. (2002). Computer go. Artificial Intelligence, 134(1):145-179.
  11. Nijssen, J. and Winands, M. (2011). Enhancements for multi-player monte-carlo tree search. Computers and Games, pages 238-249.
  12. Papadimitriou, C. M. (1994). Computational complexity. Addison-Wesley, Reading, Massachusetts.
  13. Ponsen, M., Gerritsen, G., and Chaslot, G. (2010). Integrating opponent models with monte-carlo tree search in poker. In Proc. Conf. Assoc. Adv. Artif. Intell.: Inter. Decis. Theory Game Theory Workshop, Atlanta, Georgia, pages 37-42.
  14. Rejeb, L., Guessoum, Z., and MHallah, R. (2005). The exploration-exploitation dilemma for adaptive agents. In Proceedings of the 5th European Workshop on Adaptive Agents and Multi-Agent Systems (AAMAS05). Citeseer.
  15. richardk (2012). The ai in the octagon theory. http://aigamedev.com/open/interview/the-octagontheory/ (last visited on 2013-05-16).
  16. Schadd, F. C. (2009). Monte-carlo search techniques in the modern board game thurn and taxis. Master's thesis, Department of Computer Science, Maastricht University.
  17. Silver, D. and Veness, J. (2010). Monte-carlo planning in large pomdps. Advances in Neural Information Processing Systems (NIPS), 46.
  18. Teytaud, F. and Teytaud, O. (2010). On the huge benefit of decisive moves in monte-carlo tree search algorithms. In Computational Intelligence and Games (CIG), 2010 IEEE Symposium on, pages 359 -364.
Download


Paper Citation


in Harvard Style

Fernandes H., Nogueira P. and Oliveira E. (2014). Monte Carlo Tree Search in The Octagon Theory . In Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-015-4, pages 328-335. DOI: 10.5220/0004757603280335


in Bibtex Style

@conference{icaart14,
author={Hugo Fernandes and Pedro Nogueira and Eugénio Oliveira},
title={Monte Carlo Tree Search in The Octagon Theory},
booktitle={Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2014},
pages={328-335},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004757603280335},
isbn={978-989-758-015-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - Monte Carlo Tree Search in The Octagon Theory
SN - 978-989-758-015-4
AU - Fernandes H.
AU - Nogueira P.
AU - Oliveira E.
PY - 2014
SP - 328
EP - 335
DO - 10.5220/0004757603280335