Using Near Infrared Spectroscopy to Index Temporal Changes in Affect in Realistic Human-robot Interactions

Megan Strait, Matthias Scheutz

2014

Abstract

Recent work in HRI found that prefrontal hemodynamic activity correlated with participants’ aversions to certain robots. Using a combination of brain-based objective measures and survey-based subjective measures, it was shown that increasing the presence (co-located vs. remote interaction) and human-likeness of the robot engaged greater neural activity in the prefrontal cortex and severely decreased preferences for future interactions. The results of this study suggest that brain-based measures may be able to capture participants’ affective responses (aversion vs. affinity), and in a variety of interaction settings. However, the brain-based evidence of this work is limited to temporally-brief (6-second) post-interaction samples. Hence, it remains unknown whether such measures can capture affective responses over the course of the interactions (rather than post-hoc). Here we extend the previous analysis to look at changes in brain activity over the time course of more realistic human-robot interactions. In particular, we replicate the previous findings, and moreover find qualitative evidence suggesting the measurability of fluctuations in affect over the course of the full interactions.

References

  1. Andujar, M., Ekandem, J. I., Gilbert, J. E., and Morreale, P. (2013). Evaluating engagement physiologically and knowledge retention subjectively through two different learning techniques. In HCI International.
  2. Broadbent, E., Kuman, V., Li, X., Sollers, J., Stafford, R. Q., MacDonald, B. A., and Wegner, D. M. (2013). Robots with display screens: a robot with a more humanlike face display is perceived to have more mind and a better personality. PLOS ONE, 8(8):e72589.
  3. Canning, C. and Scheutz, M. (2013). Functional nearinfrared spectroscopy in human-robot interaction. Journal of Human-Robot Interaction, 2(3):62-84.
  4. Chaminade, T., Zecca, M., Blakemore, S., Takanishi, A., Frith, C. D., Micera, S., Dario, P., Rizzolatti, G., Gallese, V., and Umilta, M. A. (2010). Brain response to a humanoid robot in areas implicated in the perception of human emotional gestures. PLoS one, 5(7).
  5. Cui, X., Bray, S., Bryant, D. M., Glover, G. H., and Reiss, A. L. (2011). A quantitative comparison of NIRS and fMRI across multiple cognitive tasks. NeuroImage, 54(4):2808-2821.
  6. Heger, D., Mutter, R., Herff, C., Putze, F., and Schultz, T. (2013). Continuous recognition of affective states by functional near infrared spectroscopy signals. In Proc. ACII.
  7. Heger, D., Putze, F., and Schultz, T. (2011). Online recognition of facial actions for natural EEG-based BCI applications. In Proc. ACII, pages 436-446.
  8. Hoshi, Y. (2011). Functional near-infrared spectroscopy: current status and future prospects. J. of Biomed. Op., 12(6).
  9. Hoshi, Y. and Tamura, M. (1997). Near-infrared optical detection of sequential brain activation in the prefrontal cortex during mental tasks. NeuroImage, 5(4):292-7.
  10. Kawaguchi, Y., Wada, K., Okamoto, M., Tsujii, T., Shibata, T., and Sakatani, K. (2012). Investigation of brain activity after interaction with seal robot measured by fNIRS. In IEEE International Symposium on Robot and Human Interactive Communication.
  11. Lisetti, C. L. (2011). Believable agents, engagement, and health interventions. In Proceedings of the Human Computer Interaction International Conference.
  12. Ochsner, K. N., Silvers, J. A., and Buhle, J. T. (2012). Functional imaging studies of emotion regulation: a synthetic review and evolving model of cognitive control of emotion. An. of NY Acad. Sci., 1251:1-24.
  13. Rosenthal von der Putten, A. M., Kramer, N. C., Hoffmann, L., Sobieraj, S., and Eimler, S. C. (2013a). An experimental study on emotional reactions towards a robot. International Journal of Social Robotics, 5:17-34.
  14. Rosenthal von der Putten, A. M., Schulte, F. P., Eimler, S. C., Hoffmann, L., Sobieraj, S., Maderwald, S., Kramer, N. C., and Brand, M. (2013b). Neural correlates of empathy towards robots. In Proc. HRI, pages 215-216.
  15. Shibata, T. (2012). Therapeutic seal robot as biofeedback medical device: Qualitative and quantitative evaluations of robot therapy in dementia care. Proceedings of the IEEE, 100(8):2527 -2538.
  16. Solovey, E. T., Girouard, A., Chauncey, K., Hirshfield, L. M., Sassaroli, A., Zheng, F., Fantini, S., and Jacob, R. J. K. (2009). Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines. In Proc. UIST.
  17. Solovey, E. T., Lalooses, F., Chauncey, K., Weaver, D., Parasi, M., Scheutz, M., Sassaroli, A., Fantini, S., Schermerhorn, P., Girouard, A., and Jacob, R. J. K. (2011). Sensing cognitive multitasking for a brainbased adaptive user interface. In Proc. CHI.
  18. Solovey, E. T., Schermerhorn, P., Scheutz, M., Sassaroli, A., Fantini, S., , and Jacob, R. J. K. (2012). Brainput: enhancing interactive systems with streaming fnirs brain input. In Proc. CHI.
  19. Strait, M., Briggs, G., and Scheutz, M. (2013a). Neural correlates of agency ascription and its role in emoStrait, M., Canning, C., and Scheutz, M. (2013b). Limitations of NIRS-based BCI for realistic applications in human-computer interaction. In 5th International BCI Meeting, pages 6-7.
  20. Strait, M., Canning, C., and Scheutz, M. (2014). Investigating the effects of robot communication strategies in advice-giving situations based on robot appearance, interaction modality, and distance. In Conference on Human-Robot Interaction (under review).
  21. Strait, M. and Scheutz, M. (2014). Measuring users' responses to humans, robots, and human-like robots with functional near infrared spectroscopy. In Conference on Human Factors in Computing (under review).
  22. Szafir, D. and Mutlu, B. (2012). Pay attention!: designing adaptive agents that monitor and improve user engagement. In Proceedings of CHI, pages 11-20.
  23. Szafir, D. and Mutlu, B. (2013). Artful: Adaptive review technology for flipped learning. In Proc. of CHI.
  24. Urry, H. L., van Reekum, C. M., Johnstone, T., Kalin, N. H., Thurow, M. E., Schaefer, H. S., Jackson, C. A., Frye, C. J., Geischar, L. L., Alexander, A. L., and Davidson, R. J. (2006). Amygdala and ventromedial prefrontal cortex are inversely coupled during regulation of negative affect and predict the diurnal pattern of cortisol secretion among older adults. J. Neurosci., 26(16):4415-4425.
  25. Zander, T. O. (2009). Detecting affective covert user states with passive brain-computer interfaces. In Proceeding of Affective Computing and Intelligent Interaction.
  26. Zander, T. O. and Kothe, C. (2011). Towards passive braincomputer interfaces: applying brain-computer interface technology to human-machine systems in general. Journal of Neural Engineering, 8(2).
Download


Paper Citation


in Harvard Style

Strait M. and Scheutz M. (2014). Using Near Infrared Spectroscopy to Index Temporal Changes in Affect in Realistic Human-robot Interactions . In Proceedings of the International Conference on Physiological Computing Systems - Volume 1: OASIS, (PhyCS 2014) ISBN 978-989-758-006-2, pages 385-392. DOI: 10.5220/0004902203850392


in Bibtex Style

@conference{oasis14,
author={Megan Strait and Matthias Scheutz},
title={Using Near Infrared Spectroscopy to Index Temporal Changes in Affect in Realistic Human-robot Interactions},
booktitle={Proceedings of the International Conference on Physiological Computing Systems - Volume 1: OASIS, (PhyCS 2014)},
year={2014},
pages={385-392},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004902203850392},
isbn={978-989-758-006-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Physiological Computing Systems - Volume 1: OASIS, (PhyCS 2014)
TI - Using Near Infrared Spectroscopy to Index Temporal Changes in Affect in Realistic Human-robot Interactions
SN - 978-989-758-006-2
AU - Strait M.
AU - Scheutz M.
PY - 2014
SP - 385
EP - 392
DO - 10.5220/0004902203850392