A New Technique of Policy Trees for Building a POMDP based Intelligent Tutoring System

Fangju Wang

2016

Abstract

Partially observable Markov decision process (POMDP) is a useful technique for building intelligent tutoring systems (ITSs). It enables an ITS to choose optimal tutoring actions when uncertainty exists. An obstacle to applying POMDP to ITSs is the great computational complexity in decision making. The technique of policy trees may improve the efficiency. However, the number of policy trees is normally exponential, and the cost for evaluating a tree is also exponential. The technique is still too expensive when applied to a practical problem. In our research, we develop a new technique of policy trees for better efficiency. The technique is aimed at minimizing the number of policy trees to evaluate in making a decision, and reducing the costs for evaluating individual trees. The technique is based on pedagogical orders of the contents in the instructional subject. In this paper, we first provide the background of ITS and POMDP, then describe the architecture of our POMDP based ITS, and then present our technique of policy trees for POMDP solving, and finally discuss some experimental results.

References

  1. Bloom, B. S. (1984) The 2 sigma problem: the search for methods of group instructions as effective as one-toone tutoring. In Educational Researcher, 13(6), 4-16.
  2. Carlin, A. and Zilberstein, S. (2008) Observation Compression in DEC-POMDP policy trees. In Proceedings of the 7th International Joint Conference on Autonomous Agents and Multi-agent Systems, 31-45.
  3. Cassandra, A. (1998) A survey of pomdp applications. In Working Notes of AAAI 1998 Fall Symposium on Planning with Partially Observable Markov Decision Process, 17-24.
  4. Cheung, B., Hui, L., Zhang, J., and Yiu, S. M. (2003) SmartTutor: an intelligent tutoring system in webbased adult education. In Elsevier The journal of Systems and software, 68, 11-25.
  5. Chinaei, H. R., Chaib-draa, B., and Lamontagne, L. (2012) Learning observation models for dialogue POMDPs. In Canadian AI'12 Proceedings of the 25th Canadian conference on Advances in Artificial Intelligence , Springer-Verlag Berlin, Heidelberg, 280-286.
  6. Folsom-Kovarik, J. T., Sukthankar, G., and Schatz, S. (2013) Tractable POMDP representations for intelligent tutoring systems. In ACM Transactions on Intelligent Systems and Technology (TIST) - Special section on agent communication, trust in multiagent systems, intelligent tutoring and coaching systems archive, 4(2), 29.
  7. Rafferty, A. N., Brunskill, E., Thomas, L., Griffiths, T. J., and Shafto, P. (2011) Faster Teaching by POMDP Planning. In Proceesings of Artificial Intelligence in Education (AIED) 2011), 280-287.
  8. Theocharous, G., Beckwith, R., Butko, N., and Philipose, M. (2009) Tractable POMDP planning algorithms for optimal teaching in SPAIS. In IJCAI PAIR Workshop (2009).
  9. VanLehn, K., van de Sande, B., Shelby, R., and Gershman, S. (2010) The Andes physics tutoring system: an experiment in Freedom. In Nkambou et-al eds. Advances in Intelligent Tutoring Systems. Berlin Heidelberg: Springer-Verlag, 421-443.
  10. Wang, F. (2015) Handling Exponential State Space in a POMDP-Based Intelligent Tutoring System. In Proceedings of 6th International Conference on E-Service and Knowledge Management (IIAI ESKM 2015), 67- 72.
  11. Williams, J. D., Poupart, P., and Young, S. (2005) Factored Partially Observable Markov Decision Processes for Dialogue Management. In Proceedings of Knowledge and Reasoning in Practical Dialogue Systems.
  12. Williams, J. D. and Young, S. (2007) Partially observable Markov decision processes for spoken dialog systems. In Elsevier Computer Speech and Language, 21, 393- 422.
  13. Woolf, B. P. (2009) Building Intelligent Interactive Tutors. Burlington, MA, USA: Morgan Kaufmann Publishers.
Download


Paper Citation


in Harvard Style

Wang F. (2016). A New Technique of Policy Trees for Building a POMDP based Intelligent Tutoring System . In Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-179-3, pages 85-93. DOI: 10.5220/0005796600850093


in Bibtex Style

@conference{csedu16,
author={Fangju Wang},
title={A New Technique of Policy Trees for Building a POMDP based Intelligent Tutoring System},
booktitle={Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU,},
year={2016},
pages={85-93},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005796600850093},
isbn={978-989-758-179-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - A New Technique of Policy Trees for Building a POMDP based Intelligent Tutoring System
SN - 978-989-758-179-3
AU - Wang F.
PY - 2016
SP - 85
EP - 93
DO - 10.5220/0005796600850093