Hybrid POMDP-BDI - An Agent Architecture with Online Stochastic Planning and Desires with Changing Intensity Levels

Gavin Rens, Thomas Meyer

2015

Abstract

Partially observable Markov decision processes (POMDPs) and the belief-desire-intention (BDI) framework have several complimentary strengths. We propose an agent architecture which combines these two powerful approaches to capitalize on their strengths. Our architecture introduces the notion of intensity of the desire for a goal’s achievement. We also define an update rule for goals’ desire levels. When to select a new goal to focus on is also defined. To verify that the proposed architecture works, experiments were run with an agent based on the architecture, in a domain where multiple goals must continually be achieved. The results show that (i) while the agent is pursuing goals, it can concurrently perform rewarding actions not directly related to its goals, (ii) the trade-off between goals and preferences can be set effectively and (iii) goals and preferences can be satisfied even while dealing with stochastic actions and perceptions. We believe that the proposed architecture furthers the theory of high-level autonomous agent reasoning.

References

  1. Antos, D. and Pfeffer, A. (2011). Using emotions to enhance decision-making. In Walsh, T., editor, Proceedings of the 22nd Intl. Joint Conf. on Artif. Intell. (IJCAI-11), pages 24-30, Menlo Park, CA. AAAI Press.
  2. Boutilier, C., Reiter, R., Soutchanski, M., and Thrun, S. (2000). Decision-theoretic, high-level agent programming in the situation calculus. In Proceedings of the Seventeenth National Conference on Artificial Intelligence (AAAI-00) and of the Twelfth Conference on Innovative Applications of Artificial Intelligence (IAAI00), pages 355-362. AAAI Press, Menlo Park, CA.
  3. Bratman, M. (1987). Intention, Plans, and Practical Reason. Harvard University Press, Massachusetts/England.
  4. Cai, C., Liao, X., and Carin, L. (2009). Learning to explore and exploit in pomdps. In NIPS, pages 198-206.
  5. Chen, Y., Hong, J., Liu, W., Godo, L., Sierra, C., and Loughlin, M. (2013). Incorporating PGMs into a BDI architecture. In Boella, G., Elkind, E., Savarimuthu, B., Dignum, F., and Purvis, M., editors, PRIMA 2013: Principles and Practice of Multi-Agent Systems, volume 8291 of Lecture Notes in Computer Science, pages 54-69. Springer, Berlin/Heidelberg.
  6. Kaelbling, L., Littman, M., and Cassandra, A. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1-2):99-134.
  7. Kinny, D. and Georgeff, M. (1991). Commitment and effectiveness of situated agents. In Proceedings of the 12th Intl. Joint Conf. on Artificial Intelligence (IJCAI-91), pages 82-88.
  8. Kinny, D. and Georgeff, M. (1992). Experiments in optimal sensing for situated agents. In Proceedings of the the 2nd Pacific Rim Intl. Conf. on Artificial Intelligence (PRICAI-92).
  9. Koenig, S. (2001). Agent-centered search. Artificial Intelligence Magazine, 22:109-131.
  10. Li, X., Cheung, W., and Liu, J. (2005). Towards solving large-scale POMDP problems via spatio-temporal belief state clustering. In Proceedings of IJCAI-05 Workshop on Reasoning with Uncertainty in Robotics (RUR-05).
  11. Lim, M., Dias, J., Aylett, R., and Paiva, A. (2008). Improving adaptiveness in autonomous characters. In Prendinger, H., Lester, J., and Ishizuka, M., editors, Intelligent Virtual Agents, volume 5208 of Lecture Notes in Computer Science, pages 348-355. Springer, Berlin/Heidelberg.
  12. Lovejoy, W. (1991). A survey of algorithmic methods for partially observed Markov decision processes. Annals of Operations Research, 28:47-66.
  13. Meneguzzi, F., Zorzo, A., Móra, M., and M., L. (2007). Incorporating planning into BDI systems. Scalable Computing: Practice and Experience, 8(1):15-28.
  14. Monahan, G. (1982). A survey of partially observable Markov decision processes: Theory, models, and algorithms. Management Science, 28(1):1-16.
  15. Murphy, R. (2000). Introduction to AI Robotics. MIT Press, Massachusetts/England.
  16. Nair, R. and Tambe, M. (2005). Hybrid bdi-pomdp framework for multiagent teaming. J. Artif. Intell. Res.(JAIR), 23:367-420.
  17. Paquet, S., Tobin, L., and Chaib-draa, B. (2005). Real-time decision making for large POMDPs. In Advances in Artificial Intelligence: Proceedings of the Eighteenth Conference of the Canadian Society for Computational Studies of Intelligence, volume 3501 of Lecture Notes in Computer Science, pages 450-455. Springer Verlag.
  18. Pereira, D., Gonc¸alves, L., Dimuro, G., and Costa, A. (2008). Constructing bdi plans from optimal pomdp policies, with an application to agentspeak programming. In G. Henning, M. G. and Goneet, S., editors, XXXIV Confereˆncia Latinoamericano de Informática, Santa Fe. Anales CLEI 2008, pages 240-249.
  19. Pollack, M. and Ringuette, M. (1990). Introducing the Tileworld: Experimentally evaluating agent architectures. In Proceedings of the AAAI-90, pages 183-189. AAAI Press.
  20. Rao, A. and Georgeff, M. (1995). BDI agents: From theory to practice. In Proceedings of the ICMAS-95, pages 312-319. AAAI Press.
  21. Rens, G., Ferrein, A., and Van der Poel, E. (2009). A BDI agent architecture for a POMDP planner. In Lakemeyer, G., Morgenstern, L., and Williams, M.- A., editors, Proceedings of the 9th Intl. Symposium on Logical Formalizations of Commonsense Reasoning (Commonsense 2009), pages 109-114, University of Technology, Sydney. UTSe Press.
  22. Ross, S., Pineau, J., Paquet, S., and Chaib-draa, B. (2008). Online planning algorithms for POMDPs. Journal of Artificial Intelligence Research (JAIR), 32:663-704.
  23. Roy, N., Gordon, G., and Thrun, S. (2005). Finding approximate POMDP solutions through belief compressions. Journal of Artificial Intelligence Research (JAIR), 23:1-40.
  24. Schut, M. and Wooldridge, M. (2000). Intention reconsideration in complex environments. In Proceedings of the the 4th Intl. Conf. on Autonomous Agents (AGENTS00), pages 209-216, New York, NY, USA. ACM.
  25. Schut, M. and Wooldridge, M. (2001a). The control of reasoning in resource-bounded agents. The Knowledge Engineering Review, 16(3):215-240.
  26. Schut, M. and Wooldridge, M. (2001b). Principles of intention reconsideration. In Agents 2001: Proceedings of the 5th Intl. Conf. on Autonomous Agents, pages 340- 347, New York, NY. ACM Press.
  27. Schut, M., Wooldridge, M., and Parsons, S. (2004). The theory and practice of intention reconsideration. Experimental and Theoretical Artificial Intelligence, 16(4):261-293.
  28. Shani, G., Brafman, R., and Shimony, S. (2007). Forward search value iteration for POMDPs. In de Mantaras, R. L., editor, Proceedings of the 20th Intl. Joint Conf. on Artif. Intell. (IJCAI-07), pages 2619-2624, Menlo Park, CA. AAAI Press.
  29. Shani, G., Pineau, J., and Kaplow, R. (2013). A survey of point-based pomdp solvers. Autonomous Agents and Multi-Agent Systems, 27(1):1-51.
  30. Simari, G. and Parsons, S. (2006). On the relationship between mdps and the bdi architecture. In Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems, AAMAS 7806, pages 1041-1048, New York, NY, USA. ACM.
  31. Simari, G. and Parsons, S. (2011). Markov Decision Processes and the Belief-Desire-Intention Model. Springer Briefs in Computer Science. Springer, New York, Dordrecht, Heidelberg, London.
  32. Walczak, A., Braubach, L., Pokahr, A., and Lamersdorf, W. (2007). Augmenting BDI agents with deliberative planning techniques. In Bordini, R., Dastani, M., Dix, J., and Seghrouchni, A., editors, Proceedings of the 4th Intl. Workshop of Programming MultiAgent Systems (ProMAS-06), pages 113-127, Heidelberg/Berlin. Springer Verlag.
  33. Wooldridge, M. (1999). Intelligent agents. In Weiss, G., editor, Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence, chapter 1. MIT Press, Massachusetts/England.
  34. Wooldridge, M. (2000). Reasoning about Rational Agents. MIT Press, Massachusetts/England.
  35. Wooldridge, M. (2002). An introduction to multiagent systems. John Wiley & Sons, Chichester, England.
Download


Paper Citation


in Harvard Style

Rens G. and Meyer T. (2015). Hybrid POMDP-BDI - An Agent Architecture with Online Stochastic Planning and Desires with Changing Intensity Levels . In Proceedings of the International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-073-4, pages 5-14. DOI: 10.5220/0005185000050014


in Bibtex Style

@conference{icaart15,
author={Gavin Rens and Thomas Meyer},
title={Hybrid POMDP-BDI - An Agent Architecture with Online Stochastic Planning and Desires with Changing Intensity Levels},
booktitle={Proceedings of the International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2015},
pages={5-14},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005185000050014},
isbn={978-989-758-073-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - Hybrid POMDP-BDI - An Agent Architecture with Online Stochastic Planning and Desires with Changing Intensity Levels
SN - 978-989-758-073-4
AU - Rens G.
AU - Meyer T.
PY - 2015
SP - 5
EP - 14
DO - 10.5220/0005185000050014