Progressive Co-adaptation in Human-Machine Interaction

Paolo Gallina, Nicola Bellotto, Massimiliano Di Luca

2015

Abstract

In this paper we discuss the concept of co-adaptation between a human operator and a machine interface and we summarize its application with emphasis on two different domains, teleoperation and assistive technology. The analysis of the literature reveals that only in a few cases the possibility of a temporal evolution of the co-adaptation parameters has been considered. In particular, it has been overlooked the role of time-related indexes that capture changes in motor and cognitive abilities of the human operator. We argue that for a more effective long-term co-adaptation process, the interface should be able to predict and adjust its parameters according to the evolution of human skills and performance. We thus propose a novel approach termed progressive co-adaptation, whereby human performance is continuously monitored and the system makes inferences about changes in the users' cognitive and motor skills. We illustrate the features of progressive co-adaptation in two possible applications, robotic telemanipulation and active vision for the visually impaired.

References

  1. Bajcsy, R. (1988). Active perception. Proc. of the IEEE, 76(8):966-1005.
  2. Bellotto, N. (2013). A multimodal smartphone interface for active perception by visually impaired. In IEEE SMC Int. Workshop on Human-Machine Systems, Cyborgs and Enhancing Devices (HUMASCEND).
  3. Bi, L., Fan, X.-A., and Liu, Y. (2013). Eeg-based braincontrolled mobile robots: A survey. IEEE Transactions on Human-Machine Systems, 43(2):161-176.
  4. Bryan, M., Martin, S., Cheung, W., and Rao, R. (2013). Probabilistic co-adaptive brain-computer interfacing. Journal of Neural Engineering, 10(6).
  5. Carlson, T. and Demiris, Y. (2012). Collaborative control for a robotic wheelchair: Evaluation of performance, attention, and workload. IEEE Trans. on Systems, Man, and Cybernetics - Part B, 42(3):876-888.
  6. Chan, L., Naghdy, F., and Stirling, D. (2014). Application of adaptive controllers in teleoperation systems: A survey. IEEE Transactions on Human-Machine Systems, 44(3):337-352.
  7. Chen, Z., Liang, B., Zhang, T., Zhang, B., and Song, H. (2014). An adaptive force reflection scheme for bilateral teleoperation. Robotica.
  8. Christoudias, C., Saenko, K., Morency, L.-P., and Darrell, T. (2006). Co-adaptation of audio-visual speech and gesture classifiers. In ICMI'06: 8th International Conference on Multimodal Interfaces, pages 84-91.
  9. Cui, Y. and Hua, J. (2013). Human behavior characteristics analysis in teleoperation system. Applied Mechanics and Materials, 373-375:163-166.
  10. Dixon, J. (2012). Human Factors in Reliable Design, pages 137-155. John Wiley & Sons, Inc.
  11. Furuta, K. (2003). Control of pendulum: From super mechano-system to human adaptive mechatronics. In Proceedings of the IEEE Conference on Decision and Control, volume 2, pages 1498-1507.
  12. Furuta, K., Kado, Y., Shiratori, S., and Suzuki, S. (2011). Assisting control for pendulum-like juggling in human adaptive mechatronics. Proceedings of the Institution of Mechanical Engineers. Part I: Journal of Systems and Control Engineering, 225(6):709-720.
  13. Gürel, T. and Mehring, C. (2012). Unsupervised adaptation of brain-machine interface decoders. Frontiers in Neuroscience, 6(16).
  14. Harashima, F. and Suzuki, S. (2010). State-of-the-art intelligent mechatronics in human-machine interaction. IEEE Industrial Electronics Magazine, 4(2):9-13.
  15. Hoey, J., Poupart, P., von Bertoldi, A., Craig, T., Boutilier, C., and Mihailidis, A. (2010). Automated handwashing assistance for persons with dementia using video and a partially observable markov decision process. Computer Vision and Image Understanding, 114(5):503-519.
  16. Hokayem, P. and Spong, M. (2006). Bilateral teleoperation: An historical survey. Automatica, 42(12):2035-2057.
  17. Huysmans, M., de Looze, M., Hoozemans, M., van der Beek, A., and van Dieen, J. (2006). The effect of joystick handle size and gain at two levels of required precision on performance and physical load on crane operators. Ergonomics, 49(11):1021-1035.
  18. Igarashi, H., Takeya, A., Kubo, Y., Suzuki, S., Harashima, F., and Kakikura, M. (2005). Human adaptive gui design for teleoperation system. In IECON Proceedings (Industrial Electronics Conference), pages 1973- 1978.
  19. Ishikuro, K., Urakawa, S., Takamoto, K., Ishikawa, A., Ono, T., and Nishijo, H. (2014). Cerebral functional imaging using near-infrared spectroscopy during repeated performances of motor rehabilitation tasks tested on healthy subjects. Frontiers in Human Neuroscience, 8(MAY).
  20. Jin, X., Zhang, J., and Liu, Y. (2014). The ergonomics research of the joystick in excavator cab. Applied Mechanics and Materials, 494-495:128-131.
  21. Jokela, T., Iivari, N., Matero, J., and Karukka, M. (2003). The standard of user-centered design and the standard definition of usability: Analyzing iso 13407 against iso 9241-11. In Proceedings of the Latin American Conference on Human-computer Interaction, CLIHC 7803, pages 53-60.
  22. Love, L. and Book, W. (2004). Force reflecting teleoperation with adaptive impedance control. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 34(1):159-165.
  23. Mavridis, N., Pierris, G., Gallina, P., Papamitsiou, Z., Astaras, A., and Moustakas, N. (2015a). On the subjective difficulty of joystick-based robot arm teleoperation with auditory feedback. In Proc. of 8th IEEE GCC Conference and Exhibition.
  24. Mavridis, N., Pierris, G., Gallina, P., Papamitsiou, Z., Astaras, A., and Moustakas, N. (2015b). Subjective difficulty and indicators of performance of joystick-based robot arm teleoperation with auditory feedback. In Proc. of IEEE International Conference on Robotics and Automation (ICRA).
  25. McAree, P. and Daniel, R. (2000). Stabilizing impacts in force-reflecting teleoperation using distance-toimpact estimates. International Journal of Robotics Research, 19(4):349-364.
  26. Merel, J., Fox, R., Jebara, T., and Paninski, L. (2013). A multi-agent control framework for co-adaptation in brain-computer interfaces. In Advances in Neural Information Processing Systems.
  27. Oliver, M., Rogers, R., Rickards, J., Tingley, M., and Biden, E. (2006). Effect of stiffness and movement speed on selected dynamic torque characteristics of hydraulicactuation joystick controls for heavy vehicles. Ergonomics, 49(3):249-268.
  28. Passenberg, C., Peer, A., and Buss, M. (2010). A survey of environment-, operator-, and task-adapted controllers for teleoperation systems. Mechatronics, 20(7):787- 801.
  29. Rivlin, E. and Rotstein, H. (2000). Control of a camera for active vision: Foveal vision, smooth tracking and saccade. International Journal of Computer Vision, 39:8-96.
  30. Sawaragi, T. (2005). Dynamical and complex behaviors in human-machine co-adaptive systems. In IFAC Proceedings Volumes (IFAC-PapersOnline), volume 16, pages 94-99.
  31. Suzuki, S., Igarashi, H., Kobayashi, H., Yasuda, T., and Harashima, F. (2013). Human adaptive mechatronics and human-system modelling. International Journal of Advanced Robotic Systems, 10.
  32. Suzuki, Y., Takase, H., Pan, Y., Ishikawa, J., and Furuta, K. (2008). Learning process of bimanual coordination. In 2008 International Conference on Control, Automation and Systems, ICCAS 2008, pages 2830- 2835.
  33. Ward, J. and Meijer, P. (2010). Visual experiences in the blind induced by an auditory sensory substitution device. Consciousness and Cognition, 19(1):492-500.
Download


Paper Citation


in Harvard Style

Gallina P., Bellotto N. and Di Luca M. (2015). Progressive Co-adaptation in Human-Machine Interaction . In Proceedings of the 12th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO, ISBN 978-989-758-123-6, pages 362-368. DOI: 10.5220/0005561003620368


in Bibtex Style

@conference{icinco15,
author={Paolo Gallina and Nicola Bellotto and Massimiliano Di Luca},
title={Progressive Co-adaptation in Human-Machine Interaction},
booktitle={Proceedings of the 12th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,},
year={2015},
pages={362-368},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005561003620368},
isbn={978-989-758-123-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,
TI - Progressive Co-adaptation in Human-Machine Interaction
SN - 978-989-758-123-6
AU - Gallina P.
AU - Bellotto N.
AU - Di Luca M.
PY - 2015
SP - 362
EP - 368
DO - 10.5220/0005561003620368