CONTROLLING A VIRTUAL BODY BY THOUGHT IN A HIGHLY-IMMERSIVE VIRTUAL ENVIRONMENT - A Case Study in Using a Brain-Computer Interface in a Virtual-Reality Cave-like System

Doron Friedman, Robert Leeb, Larisa Dikovsky,, Miriam Reiner, Gert Pfurtscheller, Mel Slater

Abstract

A brain-computer interface (BCI) can arguably be considered the ultimate user interface, where humans operate a computer using thought alone. We have integrated the Anon-BCI into a highly immersive Cave-like system. In this paper we report a case study where three participants were able to control their avatar using only their thought. We have analyzed the participants’ subjective experience using an in-depth qualitative methodology. We also discuss some limitations of BCI in controlling a virtual environment, and interaction design decisions that needed to be made.

References

  1. Armel, K. C. and Ramachandran, V. S. (2003). Projecting sensations to external objects: Evidence from skin conductance response. Proc R Soc Lond B Biol Sci, 270:1499-1506.
  2. Bayliss, J. D. (2003). Use of the evoked potential P3 component for control in a virtual apartment. IEEE Trans. Rehabil. Eng., 11(2):113-116.
  3. Bayliss, J. D. and Ballard, D. H. (2000). A virtual reality testbed for Brain Computer Interface research. IEEE Trans. Rehabil. Eng., 8(2):188-190.
  4. Botvinick, M. and Cohen, J. (1998). Rubber hands 'feel' touch that eyes see. Nature, 391(6669):756.
  5. Cruz-Neira, C., Sandin, D. J., DeFanti, T. A., Kenyon, R. V., and Hart, J. C. (1992). The CAVE: Audio visual experience automatic virtual environment. Comm. ACM, 35(6):65-72.
  6. Desmond, D., Horgan, O., and MacLachlan, M. (2002). Trinity Assessment of Body Plasticity (TABP). Department of Psychology, Trinity College, Dublin.
  7. Frecon, E., Smith, G., Steed, A., Stenius, M., and Stahl, O. (2001). An overview of the COVEN platform. Presence: Teleoperators and Virtual Environments, 10(1):109-127.
  8. Friedman, D., Leeb, R., Guger, C., Steed, A., Pfertscheller, G., and Slater, M. (2006). Navigating virtual reality by thought: What is it like? to appear in: Teleoperators and virtual environments.
  9. Gibson, W. (1984). Neuromancer. Voyager.
  10. Guger, C., Edlinger, G., Harkam, W., Niedermayer, I., and Pfurtscheller, G. (2003). How many people are able to operate an EEG-based brain-computer interface (BCI)? IEEE Trans. Neural Syst. Rehabil. Eng., 11:145-147.
  11. Kelly, F. D. and Main, F. O. (1978). Ideographic research in individual psychology: Problems and solutions. The Journal of Individual Psychology, 34(2).
  12. Middendorf, M., McMillan, G., Calhoun, G., and Jones, K. S. (2000). Brain-Computer Interface based on the steady-state visual-evoked response. IEEE Trans. Rehabil. Eng., 8(2):211-214.
  13. Millan, J. D. R. and Mourino, J. (2003). Asynchronous BCI and local neural classifiers: An overview of the adaptive brain interface project. IEEE Trans. Neural Syst. Rehabil. Eng., 11:159-161.
  14. Nelson, W., Hettinger, L., Cunningham, J., and Roe, M. (1997). Navigating through virtual flight environments using brain-body-actuated control. In Proc. IEEE Virtual Reality Annual Intl Symp., pages 30-37.
  15. Pandzic, I. S., Thalmann, N. M., Capin, T. K., and Thalmann, D. (1997). Virtual life network: A bodycentered networked virtual environment. PresenceTeleoperators and Virtual Environments, 6:676-686.
  16. Pfurtscheller, G. and Neuper, C. (2001). Motor imagery and direct brain computer communication. Proc. of the IEEE, 89(7):1123-1134.
  17. Riva, G., Davide, F., and IJsselsteijn, W., editors (2003). Being There: Concepts, effects and measurement of user presence in synthetic environments. Ios Press.
  18. Schneiderman, B. (1983). Direct manipulation: A step beyond programming languages. IEEE Computer, 16(8):57-69.
  19. Slater, M., Steed, A., McCarthy, J., and Maringelli, F. (1998). The influence of body movement on subjective presence in virtual environments. Human Factors, 40:469-477.
  20. Slater, M., Usoh, M., and Steed, A. (1994). Depth of presence in virtual environments. Presence: Teleoperators and Virtual Environments, 11(3):130-144.
  21. Smith, J. (1977). Semi-structured interviewing and qualitative analysis. In Smith, J., Harre, R., and Langenhove, L., editors, Rethinking Methods in Psychology, pages 9-26. Sage Publications, London.
  22. Steed, A., Mortensen, J., and Frecon, E. (2001). Spelunking: Experiences using the DIVE system on CAVElike platforms. In Immersive Projection Technologies and Virtual Environments, volume 2, pages 153-164. Springer-Verlag/Wien.
  23. Stephenson, N. (1991). Snowcrash. ROC Publishing.
  24. Sutherland, I. (1965). The ultimate display. In Proc. IFIPS Congress 1965, volume 2, pages 506-508.
  25. Vidal, J. J. (1973). Toward direct brain-computer communication. Annual Review of Biophysics and Bioengineering, 2:157-180.
  26. Vives, M. S. and Slater, M. (2005). From presence to consciousness through virtual reality. Nature Reviews Neuroscience, 6(4):332-339.
  27. Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., and Vaughan, T. M. (2002). Brain Computer Interfaces for communication and control. Clin. Neurophysiol., 113(6):767-791.
Download


Paper Citation


in Harvard Style

Friedman D., Leeb R., Dikovsky, L., Reiner M., Pfurtscheller G. and Slater M. (2007). CONTROLLING A VIRTUAL BODY BY THOUGHT IN A HIGHLY-IMMERSIVE VIRTUAL ENVIRONMENT - A Case Study in Using a Brain-Computer Interface in a Virtual-Reality Cave-like System . In Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP, ISBN 978-972-8865-72-6, pages 83-90. DOI: 10.5220/0002073100830090


in Bibtex Style

@conference{grapp07,
author={Doron Friedman and Robert Leeb and Larisa Dikovsky, and Miriam Reiner and Gert Pfurtscheller and Mel Slater},
title={CONTROLLING A VIRTUAL BODY BY THOUGHT IN A HIGHLY-IMMERSIVE VIRTUAL ENVIRONMENT - A Case Study in Using a Brain-Computer Interface in a Virtual-Reality Cave-like System},
booktitle={Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP,},
year={2007},
pages={83-90},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002073100830090},
isbn={978-972-8865-72-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP,
TI - CONTROLLING A VIRTUAL BODY BY THOUGHT IN A HIGHLY-IMMERSIVE VIRTUAL ENVIRONMENT - A Case Study in Using a Brain-Computer Interface in a Virtual-Reality Cave-like System
SN - 978-972-8865-72-6
AU - Friedman D.
AU - Leeb R.
AU - Dikovsky, L.
AU - Reiner M.
AU - Pfurtscheller G.
AU - Slater M.
PY - 2007
SP - 83
EP - 90
DO - 10.5220/0002073100830090