Utterance Behavior of Users While Playing Basketball with a Virtual Teammate

Divesh Lala, Yuanchao Li, Tatsuya Kawahara


Research on human-agent interaction has focused mainly on domains which are conversational in nature, but little work has been done on examining the behavior of interactive agents in domains such as team sports. This paper analyzes utterance behavior in this domain, specifically a virtual basketball game with an agent teammate. The main motivation is to assess the nature of utterances during the course of a game. We use a Wizard-of-Oz system which allows a hidden operator to appropriately respond to user utterances. Utterances are analyzed by annotating and categorizing according to Searle’s illocutionary speech acts. We find that there is evidence to support the process of the user beginning with basic utterances needed to play the game, confirming that the agent can understand them, and then moving to more complex utterances. We also find that non-task utterances are used and their proportion increases as the game progresses.


  1. Aharoni, E. and Fridlund, A. J. (2007). Social reactions toward people vs. computers: How mere lables shape interactions. Computers in human behavior, 23(5):2175-2189.
  2. Bartneck, C., Kulic, D., Croft, E., and Zoghbi, S. (2009). Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International Journal of Social Robotics, 1(1):71-81.
  3. Baur, T., Damian, I., Gebhard, P., Porayska-Pomsta, K., and Andre, E. (2013). A job interview simulation: Social cue-based interaction with a virtual character. In 2013 International Conference on Social Computing (SocialCom), pages 220-227.
  4. Bickmore, T. and Cassell, J. (2005). Social dialongue with embodied conversational agents. In Advances in natural multimodal dialogue systems, pages 23-54. Springer.
  5. Bickmore, T., Pfeifer, L., and Schulman, D. (2011). Relational agents improve engagement and learning in science museum visitors. In Vilhjálmsson, H., Kopp, S., Marsella, S., and Thórisson, K., editors, Intelligent Virtual Agents, volume 6895 of Lecture Notes in Computer Science, pages 55-67. Springer Berlin Heidelberg.
  6. Bradshaw, J. M., Feltovich, P., Johnson, M., Breedy, M., Bunch, L., Eskridge, T., Jung, H., Lott, J., Uszok, A., and van Diggelen, J. (2009). From tools to teammates: Joint activity in human-agent-robot teams. In Human Centered Design, pages 935-944. Springer.
  7. Bunt, H. (2009). The DIT++ taxonomy for functional dialogue markup. In AAMAS 2009 Workshop, Towards a Standard Markup Language for Embodied Dialogue Acts, pages 13-24.
  8. Campano, S., Durand, J., and Clavel, C. (2014). Comparative analysis of verbal alignment in human-human and human-agent interactions. In Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC 2014).
  9. Core, M. G. and Allen, J. (1997). Coding dialogs with the DAMSL annotation scheme. In AAAI fall symposium on communicative action in humans and machines, volume 56.
  10. DeVault, D., Artstein, R., Benn, G., Dey, T., Fast, E., Gainer, A., Georgila, K., Gratch, J., Hartholt, A., Lhommet, M., et al. (2014). Simsensei kiosk: A virtual human interviewer for healthcare decision support. In Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems, pages 1061-1068. International Foundation for Autonomous Agents and Multiagent Systems.
  11. Fox, J., Ahn, S. J. G., Janssen, J. H., Yeykelis, L., Segovia, K. Y., and Bailenson, J. N. (2015). Avatars versus agents: A meta-analysis quantifying the effect of agency on social influence. Human-Computer Interaction, 30(5):401-432.
  12. Gulz, A. (2005). Social enrichment by virtual characters differential benefits. Journal of Computer Assisted Learning, 21(6):405-418.
  13. Jurafsky, D., Shriberg, E., and Biasca, D. (1997). Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual. Institute of Cognitive Science Technical Report, pages 97-102.
  14. Kopp, S., Gesellensetter, L., Krämer, N. C., and Wachsmuth, I. (2005). A conversational agent as museum guide-design and evaluation of a real-world application. In Panayiotopoulos, T., Gratch, J., Aylett, R., Ballin, D., Olivier, P., and Rist, T., editors, Intelligent Virtual Agents, pages 329-343. Springer.
  15. Lala, D., Nishida, T., and Mohammad, Y. (2014). A joint activity theory analysis of body interactions in multiplayer virtual basketball. In Proceedings of the 28th International BCS Human Computer Interaction Conference on HCI 2014 - Sand, Sea and Sky - Holiday HCI, BCS-HCI 7814, pages 62-71. BCS.
  16. Langlet, C. and Clavel, C. (2014). Modelling users attitudinal reactions to the agent utterances: focus on the verbal content. In 5th International Workshop on Corpora for Research on Emotion, Sentiment & Social Signals (ES3 2014), Reykjavik, Iceland.
  17. Li, S., Sun, W., and Miller, T. (2015). Communication in human-agent teams for tasks with joint action. In COIN 2015: The XIX International Workshop on Coordination, Organizations, Institutions and Norms in Multiagent Systems, pages 111-126.
  18. Niewiadomski, R., Bevacqua, E., Mancini, M., and Pelachaud, C. (2009). Greta: An interactive expressive ECA system. In Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, pages 1399-1400. International Foundation for Autonomous Agents and Multiagent Systems.
  19. Open JTalk (2015). Open JTalk - HMM-based Text-toSpeech System. http://open-jtalk.sp.nitech.ac.jp/.
  20. Poizat, G., Bourbousson, J., Saury, J., and Sève, C. (2012). Understanding team coordination in doubles table tennis: Joint analysis of first- and third-person data. Psychology of Sport and Exercise, 13(5):630 - 639. A Sport Psychology Perspective on Olympians and the Olympic Games.
  21. Rickel, J. and Johnson, W. L. (1999). Virtual humans for team training in virtual reality. In Proceedings of the Ninth International Conference on Artificial Intelligence in Education.
  22. Rickel, J. and Johnson, W. L. (2000). Task-oriented collaboration with embodied agents in virtual worlds. Embodied conversational agents, pages 95-122.
  23. Robinson, S., Traum, D., Ittycheriah, M., and Henderer, J. (2008). What would you ask a Conversational Agent? Observations of Human-Agent Dialogues in a Museum Setting. In International Conference on Language Resources and Evaluation (LREC).
  24. Schroder, M., Bevacqua, E., Cowie, R., Eyben, F., Gunes, H., Heylen, D., ter Maat, M., McKeown, G., Pammi, S., Pantic, M., Pelachaud, C., Schuller, B., de Sevin, E., Valstar, M., and Wollmer, M. (2012). Building autonomous sensitive artificial listeners. IEEE Transactions on Affective Computing, 3(2):165-183.
  25. Searle, J. R. (1975). A taxonomy of illocutionary acts. In Gunderson, K., editor, Language, Mind and Knowledge, pages 344-369. University of Minnesota Press.
  26. Shriberg, E., Dhillon, R., Bhagat, S., Ang, J., and Carvey, H. (2004). The ICSI meeting recorder dialog act (MRDA) corpus. Technical report, DTIC Document.
  27. Taylor, N. (2012). a Silent Team is a Dead Team: Communicative norms in competitive FPS play. In Voorhees, G. A., Call, J., and Whitlock, K., editors, Guns, Grenades, and Grunts: First-person Shooter Games, pages 251-275. Bloomsbury Publishing USA.
  28. Traum, D. R. (1999). Speech acts for dialogue agents. In Foundations of rational agency, pages 169-201. Springer.
  29. Traum, D. R. (2000). 20 questions on dialogue act taxonomies. Journal of Semantics, 17(1):7-30.
  30. Travassos, B., Araújo, D., Vilar, L., and McGarry, T. (2011). Interpersonal coordination and ball dynamics in futsal (indoor football). Human Movement Science, 30(6):1245-1259.
  31. Veletsianos, G. (2012). How do learners respond to pedagogical agents that deliver social-oriented non-task messages? Impact on student learning, perceptions, and experiences. Computers in Human Behavior, 28(1):275-283.

Paper Citation

in Harvard Style

Lala D., Li Y. and Kawahara T. (2017). Utterance Behavior of Users While Playing Basketball with a Virtual Teammate . In Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-219-6, pages 28-38. DOI: 10.5220/0006119400280038

in Bibtex Style

author={Divesh Lala and Yuanchao Li and Tatsuya Kawahara},
title={Utterance Behavior of Users While Playing Basketball with a Virtual Teammate},
booktitle={Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},

in EndNote Style

JO - Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - Utterance Behavior of Users While Playing Basketball with a Virtual Teammate
SN - 978-989-758-219-6
AU - Lala D.
AU - Li Y.
AU - Kawahara T.
PY - 2017
SP - 28
EP - 38
DO - 10.5220/0006119400280038