Multi-agent Reinforcement Learning based on Multi-channel ART Networks

Hitomi Morishita, Hiroaki Ueda, Kenichi Takahashi

Abstract

3-channel fuzzy ART network FALCON is a good solution to combine reinforcement learning with state segmentation, where it learns the relations among percepts, actions and rewards. FALCON, however, does not have a mechanism to predict behavior of other agents, and thus it is difficult for FALCON to learn the optimal agent’s behavior in a multi-agent circumstance. In this paper, an action prediction module based on 2-channel fuzzy ART network is proposed, and FALCON is modified in order to be able to register the output of the action prediction module. The modified FALCON is called FALCON AP. Moreover, FALCON ER that estimates the expected value of rewards and selects an action according to the value is proposed. Through experiments in which FALCON, FALCON AP and FALCON ER are applied to a card game Hearts, it is shown that FALCON ER receives less penalty points and learns better rules.

References

  1. CST (2010). IEICE Concurrent System Technology, http://www.ieice.org/ cst/compe10/.
  2. Fujita, H. (2004). A reinforcement learning scheme for a partially-observable multi-agent system. Master's thesis, Graduate school of Information Science, Nara Institue of Science amd Technology.
  3. G. A. Carpenter, S. G. and Rosen, D. (1991). Fuzzy ART: fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Networks, Vol.4(No.6):pp.759-771.
  4. Hamagami, T. and Hirata, H. (2003). An adjustment of the number of states on Q-learning segmenting state space adaptively. Proc. of the IEEE International Conference on Systems, Man, and Cybernetics, pages 3062- 3067.
  5. Hearts (2012). Hearts for GNOME, http://www.jejik.com/ gnome-hearts/.
  6. H.Ueda, T.Naraki, Y. K. K. and T.Miyahara (2008). State space segmentation for acquisition of agent behavior. Web Intelligence and Agent Systems: An International Journal, IOS Press, Vol.6(No.4):pp.373-385.
  7. RoboCup (2012). The RoboCup Federation, http:// www.robocup.org/.
  8. Tan, A. H. (2004). FALCON: A fusion architecture for learning, cognition, and navigation. Proc. of International Joint Conference on Neural Networks, pages 3297-3302.
  9. Tan, A. H. and Xiao, D. (2005). Self-organizing cognitive agents and reinforcement learning in multi-agent environment. Proc. of the IEEE/WIC/ACM International Conference on Intelligent Agent Technology, pages 351-357.
Download


Paper Citation


in Harvard Style

Morishita H., Ueda H. and Takahashi K. (2013). Multi-agent Reinforcement Learning based on Multi-channel ART Networks . In Proceedings of the 5th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-8565-39-6, pages 461-464. DOI: 10.5220/0004239104610464


in Bibtex Style

@conference{icaart13,
author={Hitomi Morishita and Hiroaki Ueda and Kenichi Takahashi},
title={Multi-agent Reinforcement Learning based on Multi-channel ART Networks},
booktitle={Proceedings of the 5th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2013},
pages={461-464},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004239104610464},
isbn={978-989-8565-39-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Multi-agent Reinforcement Learning based on Multi-channel ART Networks
SN - 978-989-8565-39-6
AU - Morishita H.
AU - Ueda H.
AU - Takahashi K.
PY - 2013
SP - 461
EP - 464
DO - 10.5220/0004239104610464