On Interaction Quality in Human-Robot Interaction

Suna Bensch, Aleksandar Jevtić, Thomas Hellström

2017

Abstract

In many complex robotics systems, interaction takes place in all directions between human, robot, and environment. Performance of such a system depends on this interaction, and a proper evaluation of a system must build on a proper modeling of interaction, a relevant set of performance metrics, and a methodology to combine metrics into a single performance value. In this paper, existing models of human-robot interaction are adapted to fit complex scenarios with one or several humans and robots. The interaction and the evaluation process is formalized, and a general method to fuse performance values over time and for several performance metrics is presented. The resulting value, denoted interaction quality, adds a dimension to ordinary performance metrics by being explicit about the interplay between performance metrics, and thereby provides a formal framework to understand, model, and address complex aspects of evaluation of human-robot interaction.

References

  1. Adams, J. A. (2005). Human-robot interaction design: Understanding user needs and requirements. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 49, pages 447-451.
  2. Barros, P. D. and Lindeman, R. (2009). A survey of user interfaces for robot teleoperation. Retrieved from: http://digitalcommons.wpi.edu/computersciencepubs/21.
  3. Breazeal, C. and Hoffman, G. (2004). Robots that work in collaboration with people. In Proceedings of AAAI Fall Symposium on The Intersection of Cognitive Science and Robotics: From Interfaces to Intelligence.
  4. Brooks, R. (1991). Intelligence without representation. Artificial Intelligence , 47:139-159.
  5. Brooks, R. A. (1990). Elephants don't play chess. Robotics and Autonomous Systems, 6:3-15.
  6. Card, S., Moran, T., and Newell, A. (1983). The Psychology of Human-Computer Interaction. Erlbaum, Hillsdale, New Jersey.
  7. Chen, J. Y. C. (2010). Robotics operator performance in a multi-tasking environment. In Barnes, M., editor, Human-Robot Interactions in Future Military Operations. Ashgate.
  8. Dominey, P. F., Mallet, A., and Yoshida, E. (2009). Realtime spoken-language programming for cooperative interaction with a humanoid apprentice. International Journal of Humanoid Robotics, 06(02):147- 171, http://dx.doi.org/10.1142/S0219843609001711.
  9. Donmez, B., Pina, P. E., and Cummings, M. L. (2008). Evaluation criteria for human-automation performance metrics. In Proceedings of the 8th Workshop on Performance Metrics for Intelligent Systems, PerMIS 7808, pages 77-82, http://dx.doi.org/10.1145/1774674.1774687, New York, NY, USA. ACM.
  10. Drury, J. L., Scholtz, J., and Kieras, D. (2007). The potential for modeling human-robot interaction with GOMS. In Sarkar, N., editor, Human-Robot Interaction, pages 21-38. I-Tech Education and Publishing.
  11. Endsley, M. R. (1988). Design and evaluation for situation awareness enhancement. In In Proceedings of the Human Factors Society 32nd Annual Meeting, Santa Monica, CA.
  12. Feil-Seifer, D. and Mataric, M. J. (2009). Encyclopedia of Complexity and System Science, chapter HumanRobot Interaction, pages 4643-4659. Springer.
  13. Feil-Seifer, D., Skinner, K., and Mataric, M. J. (2007). Benchmarks for evaluating socially assistive robotics. Interaction Studies, 8(3):432-439.
  14. Fong, T., Thorpe, C., and Baur, C. (2003). Collaboration, dialogue, human-robot interaction. In Jarvis, R. A. and Zelinsky, A., editors, Robotics Research: The Tenth International Symposium, pages 255-266, http://dx.doi.org/10.1007/3-540-36460-9 17, Berlin, Heidelberg. Springer Berlin Heidelberg.
  15. Glas, D., Kanda, T., Ishiguro, H., and Hagita, N. (2012). Teleoperation of multiple social robots. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 42(3):530-544, http://dx.doi.org/10.1109/TSMCA.2011.2164243.
  16. Goodrich, M. A., Crandall, J. W., and Stimpson, J. L. (2003). Neglect tolerant teaming: Issues and dilemmas. In Proceedings of the 2003 AAAI Spring Symposium on Human Interaction with Autonomous Systems in Complex Environments.
  17. Goodrich, M. A. and Olsen, D. R. (2003). Seven principles of efficient human robot interaction. In Systems, Man and Cybernetics, 2003. IEEE International Conference on, volume 4, pages 3942-3948 vol.4, http://dx.doi.org/10.1109/ICSMC.2003.1244504.
  18. Goodrich, M. A. and Schultz, A. C. (2007). Human-robot interaction: A survey. Found. Trends Hum.-Comput. Interact., 1(3):203-275, http://dx.doi.org/10.1561/1100000005.
  19. Hiroi, Y. and Ito, A. (2012). Informatics in Control, Automation and Robotics: Volume 2, chapter Toward Human-Robot Interaction Design through HumanHuman Interaction Experiment, pages 127-130, http://dx.doi.org/10.1007/978-3-642-25992-0 18. Springer Berlin Heidelberg, Berlin, Heidelberg.
  20. John, B. E. and Kieras, D. E. (1996). Using GOMS for user interface design and evaluation. ACM Transactions on Human-Computer Interaction, 3(4).
  21. Kahn, P. H., Ishiguro, H., Friedman, B., Freier, N. G., Severson, R. L., and Miller, J. (2007). What is a human? toward psychological benchmarks in the field of human-robot interaction. interaction studies: Social behaviour and communication in. Biological and Artificial Systems , pages 1-5.
  22. Krämer, N. C., Pütten, A., and Eimler, S. (2012). HumanAgent and Human-Robot Interaction Theory: Similarities to and Differences from Human-Human Interaction, pages 215-240, http://dx.doi.org/10.1007/978- 3-642-25691-2 9. Springer Berlin Heidelberg, Berlin, Heidelberg.
  23. Minsky, M. (1980). Telepresence. Omni, 2(9):45-52.
  24. Norman, D. A. (2002). The Design of Everyday Things. Basic Books, Inc., New York, NY, USA.
  25. Olsen, D. R. and Goodrich, M. A. (2003). Metrics for evaluating human-robot interaction. In Proceeedings of NIST Performance Metrics for Intelligent Systems Workshop.
  26. Pfeifer, R. and Scheier, C. (2001). Understanding Intelligence. MIT Press.
  27. Scholtz, J. (2003). Theory and evaluation of human robot interactions. In System Sciences, 2003. Proceedings of the 36th Annual Hawaii International Conference on, pages 10-, http://dx.doi.org/10.1109/HICSS.2003.1174284.
  28. Scholtz, J., Antonishek, B., and Young, J. (2005). Implementation of a situation awareness assessment tool for evaluation of human-robot interfaces. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 35(4):450-459, http://dx.doi.org/10.1109/TSMCA.2005.850589.
  29. Sheridan, T. B. (1992). Telerobotics, Automation, and Human Supervisory Control. MIT Press, Cambridge, MA, USA.
  30. Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., and Goodrich, M. A. (2006). Common metrics for human-robot interaction. In Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-robot Interaction, HRI06, pages 33- 40, http://dx.doi.org/10.1145/1121241.1121249, New York, NY, USA. ACM.
  31. Taha, T., Miro, J., and Dissanayake, G. (2011). A POMDP framework for modelling human interaction with assistive robots. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, pages 544- 549, http://dx.doi.org/10.1109/ICRA.2011.5980323.
  32. Tsui, K. M., Desai, M., Yanco, H. A., and Uhlik, C. (2011). Exploring use cases for telepresence robots. In Proceedings of the 6th International Conference on Human-robot Interaction, HRI 7811, pages 11- 18, http://dx.doi.org/10.1145/1957656.1957664, New York, NY, USA. ACM.
  33. Yanco, H. and Drury, J. (2004). Classifying humanrobot interaction: an updated taxonomy. In Systems, Man and Cybernetics, 2004 IEEE International Conference on, volume 3, pages 2841-2846 , http://dx.doi.org/10.1109/ICSMC.2004.1400763.
  34. Young, J. E., Sung, J., Voida, A., Sharlin, E., Igarashi, T., Christensen, H., and Grinter, R. (2011). Evaluating human-robot interaction: Focusing on the holistic interaction experience. International Journal on Social Robotics, 3(1):53-67.
  35. Ziemke, T. (2001). Are robots embodied? In Balkenius, Zlatev, Dautenhahn, Kozima, and Breazeal, editors, Proceedings of the First International Workshop on Epigenetic Robotics - Modeling Cognitive Development in Robotic Systems, volume 85, pages 75-83, Lund, Sweden. Lund University Cognitive Studies.
Download


Paper Citation


in Harvard Style

Bensch S., Jevtić A. and Hellström T. (2017). On Interaction Quality in Human-Robot Interaction . In Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-219-6, pages 182-189. DOI: 10.5220/0006191601820189


in Bibtex Style

@conference{icaart17,
author={Suna Bensch and Aleksandar Jevtić and Thomas Hellström},
title={On Interaction Quality in Human-Robot Interaction},
booktitle={Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2017},
pages={182-189},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006191601820189},
isbn={978-989-758-219-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - On Interaction Quality in Human-Robot Interaction
SN - 978-989-758-219-6
AU - Bensch S.
AU - Jevtić A.
AU - Hellström T.
PY - 2017
SP - 182
EP - 189
DO - 10.5220/0006191601820189