# STRATEGIES FOR CHALLENGING TWO-PLAYER GAMES - Some Lessons from Iterated Traveler’s Dilemma

### Predrag T. Tošić, Philip C. Dasler

#### Abstract

We study the iterated version of the Traveler’s Dilemma (TD). TD is a two-player, non-zero sum game that offers plenty of incentives for cooperation. Our goal is to gain deeper understanding of iterated two-player games whose structures are far from zero-sum. Our experimental study and analysis of Iterated TD is based on a round-robin tournament we have recently designed, implemented and analyzed. This tournament involves 38 distinct participating strategies, and is motivated by the seminal work by Axelrod et al. on Iterated Prisoners Dilemma. We first motivate and define the strategies competing in our tournament, followed by a summary of the tournament results with respect to individual strategies. We then extend the performance comparisonand- contrast of individual strategies in the tournament, and carefully analyze how groups of closely related strategies perform when each such group is viewed as a “team”. We draw some interesting lessons from the analyzes of individual and team performances, and outline some promising directions for future work.

#### References

- Axelrod, R. (1980). Effective choice in the prisoner's dilemma. Journal of Conflict Resolution, 24(1):3 -25.
- Axelrod, R. (1981). The evolution of cooperation. Science, 211(4489):1390-1396.
- Axelrod, R. (2006). The evolution of cooperation. Basic Books.
- Basu, K. (1994). The traveler's dilemma: Paradoxes of rationality in game theory. The American Economic Review, 84(2):391-395.
- Basu, K. (2007). The traveler's dilemma. Scientific American Magazine.
- Beaufils, B., Delahaye, J.-P., and Mathieu, P. (1998). Complete classes of strategies for the classical iterated prisoner's dilemma. In Evolutionary Programming, pages 33-41.
- Becker, T., Carter, M., and Naeve, J. (2005). Experts playing the traveler's dilemma. Technical report, Department of Economics, University of Hohenheim, Germany.
- Capra, C. M., Goeree, J. K., Gmez, R., and Holt, C. A. (1999). Anomalous behavior in a traveler's dilemma? The American Economic Review, 89(3):678-690.
- Dasler, P. and Tosic, P. (2010). The iterated traveler's dilemma: Finding good strategies in games with “bad” structure: Preliminary results and analysis. In Proc of the 8th Euro. Workshop on Multi-Agent Systems, EUMAS'10.
- Dasler, P. and Tosic, P. (2011). Playing challenging iterated two-person games well: A case study on iterated travelers dilemma. In Proc. of WorldComp Foundations of Computer Science FCS'11; to appear.
- Goeree, J. K. and Holt, C. A. (2001). Ten little treasures of game theory and ten intuitive contradictions. The American Economic Review, 91(5):1402-1422.
- Halpern, J. Y. and Pass, R. (2009). Iterated regret minimization: a new solution concept. In Proceedings of the 21st international jont conference on Artifical intelligence, IJCAI'09, pages 153-158, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
- Land, S., van Neerbos, J., and Havinga, T. (2008). Analyzing the traveler's dilemma Multi-Agent systems project.
- Littman, M. L. (2001). Friend-or-Foe q-learning in GeneralSum games. In Proc. of the 18th Int'l Conf. on Machine Learning, pages 322-328. Morgan Kaufmann Publishers Inc.
- Neumann, J. V. and Morgenstern, O. (1944). Theory of games and economic behavior. Princeton Univ. Press.
- Osborne, M. (2004). An introduction to game theory. Oxford University Press, New York.
- Pace, M. (2009). How a genetic algorithm learns to play traveler's dilemma by choosing dominated strategies to achieve greater payoffs. In Proc. of the 5th international conference on Computational Intelligence and Games, pages 194-200.
- Parsons, S. and Wooldridge, M. (2002). Game theory and decision theory in Multi-Agent systems. Autonomous Agents and Multi-Agent Systems, 5:243-254.
- Rapoport, A. and Chammah, A. M. (1965). Prisoner's Dilemma. Univ. of Michigan Press.
- Rosenschein, J. S. and Zlotkin, G. (1994). Rules of encounter: designing conventions for automated negotiation among computers. MIT Press.
- Tosic, P. and Dasler, P. (2011). How to play well in non-zero sum games: Some lessons from generalized traveler's dilemma. In Zhong, N., Callaghan, V., Ghorbani, A., and Hu, B., editors, Active Media Technology, volume 6890 of Lecture Notes in Computer Science, pages 300-311. Springer Berlin / Heidelberg.
- Watkins, C. and Dayan, P. (1992). Q-learning. Machine Learning, 8(3-4):279-292.
- Wooldridge, M. (2009). An Introduction to MultiAgent Systems. John Wiley and Sons.
- Zeuthen, F. F. (1967). Problems of monopoly and economic warfare / by F. Zeuthen ; with a preface by Joseph A. Schumpeter. Routledge and K. Paul, London. First published 1930 by George Routledge & Sons Ltd.
- Random [99, 100]
- Always 100
- Always 99
- Mixed - L(y-g) E(x-g) H(x-g), 80%); (100, 20%)
- Simple Trend - K = 3, Eps = 0.5
- Mixed - TFT (y-g), 80%); (R[99, 100], 20%)
- Simple Trend - K = 10, Eps = 0.5
- Simple Trend - K = 25, Eps = 0.5
- Mixed - L(x) E(x) H(y-g), 80%); (100, 20%)
- Mixed - L(y-g) E(x-g) H(x-g), 80%); (100, 10%); (2, 10%)
- Q Learn - alpha= 0.2, discount= 0.0
- Q Learn - alpha= 0.5, discount= 0.0
- Q Learn - alpha= 0.5, discount= 0.9
- Q Learn - alpha= 0.2, discount= 0.9
- Q Learn - alpha= 0.8, discount= 0.0
- Q Learn - alpha= 0.8, discount= 0.9
- Buckets - PD, Retention = 0.5
- Always 51
- Buckets - PD, Retention = 0.2
- Buckets - PD, Retention = 0.8
- TFT - Simple (y-2g)
- Random [2, 100]
- Mixed - L(y-g) E(x-g) H(x-g), 80%); (2, 20%)
- Mixed - L(x) E(x) H(y-g), 80%); (100, 10%); (2, 10%)
- TFT - Low(x) Equal(x-2g) High(y-g)
- TFT - Low(x-2g) Equal(x) High(y-g)
- TFT - Low(x-2g) Equal(x-2g) High(y-g)
- Mixed - L(x) E(x) H(y-g), 80%); (2, 20%)
- Always 2

#### Paper Citation

#### in Harvard Style

T. Tošić P. and C. Dasler P. (2012). **STRATEGIES FOR CHALLENGING TWO-PLAYER GAMES - Some Lessons from Iterated Traveler’s Dilemma** . In *Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,* ISBN 978-989-8425-96-6, pages 72-82. DOI: 10.5220/0003753900720082

#### in Bibtex Style

@conference{icaart12,

author={Predrag T. Tošić and Philip C. Dasler},

title={STRATEGIES FOR CHALLENGING TWO-PLAYER GAMES - Some Lessons from Iterated Traveler’s Dilemma},

booktitle={Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},

year={2012},

pages={72-82},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0003753900720082},

isbn={978-989-8425-96-6},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,

TI - STRATEGIES FOR CHALLENGING TWO-PLAYER GAMES - Some Lessons from Iterated Traveler’s Dilemma

SN - 978-989-8425-96-6

AU - T. Tošić P.

AU - C. Dasler P.

PY - 2012

SP - 72

EP - 82

DO - 10.5220/0003753900720082