Learning Models of Human Behaviour from Textual Instructions

Kristina Yordanova, Thomas Kirste

2016

Abstract

There are various activity recognition approaches that rely on manual definition of precondition-effect rules to describe human behaviour. These rules are later used to generate computational models of human behaviour that are able to reason about the user behaviour based on sensor observations. One problem with these approaches is that the manual rule definition is time consuming and error prone process. To address this problem, in this paper we propose an approach that learns the rules from textual instructions. In difference to existing approaches, it is able to learn the causal relations between the actions without initial training phase. Furthermore, it learns the domain ontology that is used for the model generalisation and specialisation. To evaluate the approach, a model describing cooking task was learned and later applied for explaining seven plans of actual human behaviour. It was then compared to a hand-crafted model describing the same problem. The results showed that the learned model was able to recognise the plans with higher overall probability compared to the hand-crafted model. It also learned a more complex domain ontology and was more general than the hand-crafted model. In general, the results showed that it is possible to learn models of human behaviour from textual instructions which are able to explain actual human behaviour.

References

  1. Branavan, S. R. K., Kushman, N., Lei, T., and Barzilay, R. (2012). Learning high-level planning from text. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1, ACL 7812, pages 126-135, Stroudsburg, PA, USA. Association for Computational Linguistics.
  2. Branavan, S. R. K., Zettlemoyer, L. S., and Barzilay, R. (2010). Reading between the lines: Learning to map high-level instructions to commands. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL 7810, pages 1268- 1277, Stroudsburg, PA, USA. Association for Computational Linguistics.
  3. Chen, D. L. and Mooney, R. J. (2011). Learning to interpret natural language navigation instructions from observations. In Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI-2011) , pages 859-865.
  4. Chen, L., Hoey, J., Nugent, C., Cook, D., and Yu, Z. (2012). Sensor-based activity recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 42(6):790-808.
  5. Granger, C. W. J. (1969). Investigating Causal Relations by Econometric Models and Cross-spectral Methods. Econometrica, 37(3):424-438.
  6. Hiatt, L. M., Harrison, A. M., and Trafton, J. G. (2011). Accommodating human variability in human-robot teams through theory of mind. In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence , IJCAI'11, pages 2066-2071, Barcelona, Spain. AAAI Press.
  7. Hoey, J., Poupart, P., Bertoldi, A. v., Craig, T., Boutilier, C., and Mihailidis, A. (2010). Automated handwashing assistance for persons with dementia using video and a partially observable markov decision process. Computer Vision and Image Understanding, 114(5):503- 519.
  8. Kollar, T., Tellex, S., Roy, D., and Roy, N. (2014). Grounding verbs of motion in natural language commands to robots. In Khatib, O., Kumar, V., and Sukhatme, G., editors, Experimental Robotics, volume 79 of Springer Tracts in Advanced Robotics, pages 31-47. Springer Berlin Heidelberg.
  9. Krüger, F., Hein, A., Yordanova, K., and Kirste, T. (2015). Recognising the actions during cooking task (cooking task dataset). University Library, University of Rostock. http://purl.uni-rostock.de/rosdok/id00000116.
  10. Krüger, F., Nyolt, M., Yordanova, K., Hein, A., and Kirste, T. (2014). Computational state space models for activity and intention recognition. a feasibility study. PLoS ONE, 9(11):e109381.
  11. Krüger, F., Yordanova, K., Hein, A., and Kirste, T. (2013). Plan synthesis for probabilistic activity recognition. In Filipe, J. and Fred, A. L. N., editors, Proceedings of the 5th International Conference on Agents and Artificial Intelligence (ICAART 2013) , pages 283-288, Barcelona, Spain. SciTePress.
  12. Krüger, F., Yordanova, K., Köppen, V., and Kirste, T. (2012). Towards tool support for computational causal behavior models for activity recognition. In Proceedings of the 1st Workshop: ”Situation-Aware Assistant Systems Engineering: Requirements, Methods, and Challenges” (SeASE 2012) held at Informatik 2012 , pages 561-572, Braunschweig, Germany.
  13. Li, X., Mao, W., Zeng, D., and Wang, F.-Y. (2010). Automatic construction of domain theory for attack planning. In IEEE International Conference on Intelligence and Security Informatics (ISI), 2010, pages 65- 70.
  14. Miller, G. A. (1995). Wordnet: A lexical database for english. Commun. ACM, 38(11):39-41.
  15. Nguyen, T. A., Kambhampati, S., and Do, M. (2013). Synthesizing robust plans under incomplete domain models. In Burges, C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K., editors, Advances in Neural Information Processing Systems 26, pages 2472-2480. Curran Associates, Inc.
  16. Okeyo, G., Chen, L., Wang, H., and Sterritt, R. (2011). Ontology-based learning framework for activity assistance in an adaptive smart home. In Chen, L., Nugent, C. D., Biswas, J., and Hoey, J., editors, Activity Recognition in Pervasive Intelligent Environments, volume 4 of Atlantis Ambient and Pervasive Intelligence, pages 237-263. Atlantis Press.
  17. Perkowitz, M., Philipose, M., Fishkin, K., and Patterson, D. J. (2004). Mining models of human activities from the web. In Proceedings of the 13th International Conference on World Wide Web, WWW 7804, pages 573-582, New York, NY, USA. ACM.
  18. Philipose, M., Fishkin, K. P., Perkowitz, M., Patterson, D. J., Fox, D., Kautz, H., and Hahnel, D. (2004). Inferring activities from interactions with objects. IEEE Pervasive Computing, 3(4):50-57.
  19. Ramirez, M. and Geffner, H. (2011). Goal recognition over pomdps: Inferring the intention of a pomdp agent. In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence , volume 3 of IJCAI'11, pages 2009-2014, Barcelona, Spain. AAAI Press.
  20. Richter, S. and Westphal, M. (2010). The lama planner: Guiding cost-based anytime planning with landmarks. Journal of Artificial Intelligence Research , 39(1):127- 177.
  21. Sil, A. and Yates, E. (2011). Extracting strips representations of actions and events. In Recent Advances in Natural Language Processing, pages 1-8.
  22. Tenorth, M., Nyga, D., and Beetz, M. (2010). Understanding and executing instructions for everyday manipulation tasks from the world wide web. In IEEE International Conference on Robotics and Automation (ICRA), pages 1486-1491.
  23. Ye, J., Stevenson, G., and Dobson, S. (2014). Usmart: An unsupervised semantic mining activity recognition technique. ACM Trans. Interact. Intell. Syst., 4(4):16:1-16:27.
  24. Yordanova, K. (2015a). Discovering causal relations in textual instructions. In Recent Advances in Natural Language Processing, pages 714-720, Hissar, Bulgaria.
  25. Yordanova, K. (2015b). Time series from textual instructions for causal relations discovery (causal relations dataset). University Library, University of Rostock. http://purl.uni-rostock.de/rosdok/id00000117.
  26. Yordanova, K. and Kirste, T. (2015). A process for systematic development of symbolic models for activity recognition. ACM Transactions on Interactive Intelligent Systems, 5(4).
  27. Yordanova, K., Krüger, F., and Kirste, T. (2012). Tool support for activity recognition with computational causal behaviour models. In Proceedings of the 35th German Conference on Artificial Intelligence , pages 561-573, Saarbrücken, Germany.
  28. Yordanova, K., Nyolt, M., and Kirste, T. (2014). Strategies for reducing the complexity of symbolic models for activity recognition. In Agre, G., Hitzler, P., Krisnadhi, A., and Kuznetsov, S., editors, Artificial Intelligence: Methodology, Systems, and Applications, volume 8722 of Lecture Notes in Computer Science, pages 295-300. Springer International Publishing.
  29. Zhuo, H. H. and Kambhampati, S. (2013). Action-model acquisition from noisy plan traces. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI), pages 2444-2450, Beijing, China. AAAI.
Download


Paper Citation


in Harvard Style

Yordanova K. and Kirste T. (2016). Learning Models of Human Behaviour from Textual Instructions . In Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-172-4, pages 415-422. DOI: 10.5220/0005755604150422


in Bibtex Style

@conference{icaart16,
author={Kristina Yordanova and Thomas Kirste},
title={Learning Models of Human Behaviour from Textual Instructions},
booktitle={Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2016},
pages={415-422},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005755604150422},
isbn={978-989-758-172-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Learning Models of Human Behaviour from Textual Instructions
SN - 978-989-758-172-4
AU - Yordanova K.
AU - Kirste T.
PY - 2016
SP - 415
EP - 422
DO - 10.5220/0005755604150422