Review of the Use of Electroencephalography as an Evaluation Method for Human-Computer Interaction

Jérémy Frey, Christian Mühl, Fabien Lotte, Martin Hachet

2014

Abstract

Evaluating human-computer interaction is essential as a broadening population uses machines, sometimes in sensitive contexts. However, traditional evaluation methods may fail to combine real-time measures, an ``objective'' approach and data contextualization. In this review we look at how adding neuroimaging techniques can respond to such needs. We focus on electroencephalography (EEG), as it could be handled effectively during a dedicated evaluation phase. We identify workload, attention, vigilance, fatigue, error recognition, emotions, engagement, flow and immersion as being recognizable by EEG. We find that workload, attention and emotions assessments would benefit the most from EEG. Moreover, we advocate to study further error recognition through neuroimaging to enhance usability and increase user experience.

References

  1. Berka, C. and Levendowski, D. (2007). EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat Space Environ Med., 78(5 Suppl):B231-44.
  2. Berta, R., Bellotti, F., De Gloria, A., Pranantha, D., and Schatten, C. (2013). Electroencephalogram and Physiological Signal Analysis for Assessing Flow in Games. IEEE Trans. Comp. Intel. and AI in Games, 5(2):164-175.
  3. Blankertz, B., Tangermann, M., Vidaurre, C., Fazli, S., Sannelli, C., Haufe, S., Maeder, C., Ramsey, L., Sturm, I., Curio, G., and M üller, K.-R. (2010). The Berlin Brain-Computer Interface: Non-Medical Uses of BCI Technology. Front Neurosci, 4(December):198.
  4. Boksem, M. a. S., Meijman, T. F., and Lorist, M. M. (2005). Effects of mental fatigue on attention: an ERP study. Cogn Brain Res, 25(1):107-16.
  5. Bowman, D., Gabbard, J., and Hix, D. (2002). A survey of usability evaluation in virtual environments: classification and comparison of methods. Presence-Teleop. Virt., 11(4):404-424.
  6. Brandt, T., Dichgans, J., and Koenig, E. (1973). Differential effects of central versus peripheral vision on egocentric and exocentric motion perception. Experimental Brain Research, 491:476-491.
  7. Brouwer, A.-M., Hogervorst, M. a., van Erp, J. B. F., Heffelaar, T., Zimmerman, P. H., and Oostenveld, R. (2012). Estimating workload using EEG spectral power and ERPs in the n-back task. J. of neur. engin., 9(4):045008.
  8. Bruseberg, A. and McDonagh-Philp, D. (2002). Focus groups to support the industrial/product designer: a review based on current literature and designers' feedback. Applied ergonomics, 33(1):27-38.
  9. Chanel, G., Rebetez, C., Bétrancourt, M., and Pun, T. (2011). Emotion assessment from physiological signals for adaptation of game difficulty. IEEE T Syst. Man Cy. A, 41(6):1052-1063.
  10. Chavarriaga, R. and Millan, J. D. R. (2010). Learning from EEG error-related potentials in noninvasive brain-computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng., 18(4):381-8.
  11. Damasio, A. R. (1994). Descartes' error: emotion, reason, and the human brain.
  12. Dirican, A. C. and G ökt ürk, M. (2011). Psychophysiological measures of human cognitive states applied in human computer interaction. Procedia Computer Science, 3:1361-1367.
  13. Fairclough, S. H. (2009). Fundamentals of physiological computing. Interacting with Comp., 21(1-2):133-145.
  14. Ferrez, P. W. and Millan, J. D. R. (2008). Error-related EEG potentials generated during simulated brain-computer interaction. IEEE Trans. Biomed. Eng., 55(3):923-9.
  15. Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. J. of experimental psychology. General, 47(6):381-391.
  16. Friedman, J. (1997). On bias, variance, 0/1loss, and the curse-of-dimensionality. Data mining and knowledge discovery, 77:55-77.
  17. George, L. and Lécuyer, A. (2010). An overview of research on'passive'brain-computer interfaces for implicit human-computer interaction. In ICABB 2010.
  18. George, L., Lotte, F., Abad, R. V., and Lécuyer, A. (2011). Using Scalp Electrical Biosignals to Control an Object by Concentration and Relaxation Tasks: Design and Evaluation. In IEEE EMBS 2011.
  19. Grimes, D., Tan, D., and Hudson, S. (2008). Feasibility and pragmatics of classifying working memory load with an electroencephalograph. CHI 7808, page 835.
  20. Hamadicharef, B. (2010). BCI literature - a bibliometric study. In ISSPA 7810, volume 1, pages 626-629. IEEE.
  21. Hart, S. and Staveland, L. (1988). Development of NASATLX (Task Load Index): Results of empirical and theoretical research. In Human mental workload.
  22. Heingartner, D. (2009). Mental block. IEEE Spectrum, 46(1):42-43.
  23. Hirshfield, L., Chauncey, K., and Gulotta, R. (2009). Combining electroencephalograph and functional near infrared spectroscopy to explore users' mental workload. FAC 7809.
  24. Jankowski, J. and Hachet, M. (2013). A Survey of Interaction Techniques for Interactive 3D Environments. In Eurographics 7813.
  25. Just, M. A., Carpenter, P. a., and Miyake, A. (2003). Neuroindices of cognitive workload: Neuroimaging, pupillometric and event-related potential studies of brain work. Theoretical Issues in Ergonomics Science, 4(1-2):56-88.
  26. Kivikangas, J. M., Ekman, I., Chanel, G., Järvelä, S., Cowley, B., Henttonen, P., and Ravaja, N. (2010). Review on psychophysiological methods in game research. Proc. of 1st Nordic DiGRA.
  27. Klimesch, W., Doppelmayr, M., Russegger, H., Pachinger, T., and Schwaiger, J. (1998). Induced alpha band power changes in the human EEG and attention. Neuroscience letters, 244(2):73-6.
  28. Kohlmorgen, J., Dornhege, G., Braun, M., Blankertz, B., Mü ller, K.-R., Curio, G., Hagemann, K., Bruns, A., Schrauf, M., and Kincses, W. (2007). Improving human performance in a real operating environment through real-time mental workload detection. In Toward Brain-Computer Interfacing.
  29. Laurent, F., Valderrama, M., Besserve, M., Guillard, M., Lachaux, J.-P., Martinerie, J., and Florence, G. (2013). Multimodal information improves the rapid detection of mental fatigue. Biomed. Sig. Proc. Contr., pages 1-9.
  30. Liu, Y., Sourina, O., and Nguyen, M. (2011). Real-time EEG-based emotion recognition and its applications. In Trans. comp. science, pages 256-277. Springer.
  31. Loggia, M. L., Juneau, M., and Bushnell, M. C. (2011). Autonomic responses to heat pain: Heart rate, skin conductance, and their relation to verbal ratings and stimulus intensity. Pain, 152(3):592-8.
  32. Lorist, M. M., Klein, M., Nieuwenhuis, S., De Jong, R., Mulder, G., and Meijman, T. F. (2000). Mental fatigue and task control: planning and preparation. Psychophysiology, 37(5):614-25.
  33. Mandryk, R., Inkpen, K., and Calvert, T. (2006). Using psychophysiological techniques to measure user experience with entertainment technologies. Behav. & Inf. Tech.
  34. Mathan, S., Whitlow, S., and Feyereisen, T. (2007). WorkSense: Exploring the Feasibility of Human Factors Assessment using Electrophysiological Sensors. In 4th IACS.
  35. Matthews, G., Campbell, S. E., Falconer, S., Joyner, L. a., Huggins, J., Gilliland, K., Grier, R., and Warm, J. S. (2002). Fundamental dimensions of subjective state in performance settings: Task engagement, distress, and worry. Emotion, 2(4):315-340.
  36. Milekovic, T., Ball, T., Schulze-Bonhage, A., Aertsen, A., and Mehring, C. (2013). Detection of error related neuronal responses recorded by electrocorticography in humans during continuous movements. PloS one, 8(2).
  37. Mühl, C., Brouwer, A., van Wouwe, N., van den Broek, E. L., Nijboer, F., and Heylen, D. (2011). Modalityspecific Affective Responses and their Implications for Affective BCI. In 5th Int. BCI Conf., pages 120- 123.
  38. Mustafa, M., Lindemann, L., and Magnor, M. (2012). EEG analysis of implicit human visual perception. CHI 7812, page 513.
  39. Nacke, L., Ambinder, M., Canossa, A., Mandryk, R., and Stach, T. (2009). Game Metrics and Biometrics: The Future of Player Experience Research. Future Play.
  40. Nacke, L. E. and Lindley, C. A. (2009). Affective ludology, flow and immersion in a first-person shooter: Measurement of player experience. J. Can. Game Stud. Ass., 3(5).
  41. Nacke, L. E., Stellmach, S., and Lindley, C. A. (2010). Electroencephalographic Assessment of Player Experience: A Pilot Study in Affective Ludology. SAG, 42(5):632-655.
  42. Nieuwenhuis, S., Ridderinkhof, K. R., Blom, J., Band, G. P., and Kok, A. (2001). Error-related brain potentials are differentially related to awareness of response errors: evidence from an antisaccade task. Psychophysiology, 38(5):752-60.
  43. Nisbett, R. E. and Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3):231-260.
  44. Ogolla, J. A. (2011). Usability Evaluation: Tasks Susceptible to Concurrent Think-Aloud Protocol. Master thesis, Linköping University.
  45. Oken, B. S., Salinsky, M. C., and Elsas, S. M. (2006). Vigilance, alertness, or sustained attention: physiological basis and measurement. Clin Neurophysiol, 117(9):1885-901.
  46. Parasuraman, R. (2013). Neuroergonomics: Brain-Inspired Cognitive engineering. In The Oxford Handbook Of Cog. Engin., page 672. Oxford University Press, USA.
  47. Partala, T. and Surakka, V. (2003). Pupil size variation as an indication of affective processing. International Journal of Human-Computer Studies, 59(1-2):185-198.
  48. Picard, R. W. (1995). Affective computing. Technical Report 321, MIT Media Laboratory.
  49. Pike, M., Wilson, M., Divoli, A., and Medelyan, A. (2012). CUES: Cognitive Usability Evaluation System. EuroHCIR 7812, pages 1-4.
  50. Posner, J., Russell, J. a., and Peterson, B. S. (2005). The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev. Psychopathol., 17(3):715-34.
  51. Ravaja, N. (2009). FUGA: The Fun of Gaming: Measuring the Human Experience of Media Enjoyment. Final Activity Report. Technical report.
  52. Saavedra, C. and Bougrain, L. (2012). Processing Stages of Visual Stimuli and Event-Related Potentials. The NeuroComp/KEOpS'12 workshop, 2:1-5.
  53. Schalk, G., Wolpaw, J. R., McFarland, D. J., and Pfurtscheller, G. (2000). EEG-based communication: presence of an error potential. Clin. Neurophysiol., 111(12):2138-44.
  54. Schmidt, N. M., Blankertz, B., and Treder, M. S. (2012). Online detection of error-related potentials boosts the performance of mental typewriters. BMC neurosc., 13(1):19.
  55. Scholler, S., Bosse, S., Treder, M. S., Blankertz, B., Curio, G., M üller, K.-R., and Wiegand, T. (2012). Toward a direct measure of video quality perception using EEG. IEEE Trans. Image Process., 21(5):2619-29.
  56. Shaw, J. C. (2003). The brain's alpha rhythms and the mind. Elsevier.
  57. Slater, M., Lotto, B., Arnold, M., and Sanchez-Vives, M. (2009). How we experience immersive virtual environments: the concept of presence and its measurement. Anuario de psicología, 40(2773):193-210.
  58. Sobolewski, A., Chavarriaga, R., and Millán, J. (2013). Error Processing of Self-paced Movements. In TOBI Workshop IV, pages 137-138.
  59. Trachel, R., Brochier, T., and Clerc, M. (2013). Enhancing visuospatial attention performance with braincomputer interfaces. CHI 7813, page 1245.
  60. van de Laar, B., Gürkök, H., Bos, D. P.-O., Nijboer, F., and Nijholt, A. (2013). Brain-Computer Interfaces and User Experience Evaluation. In Towards Practical Brain-Computer Interfaces, pages 223-237. Springer.
  61. van Erp, J. B. F., Veltman, H., and Grootjen, M. (2010). Brain-Based Indices for User System Symbiosis. In Brain-Computer Interfaces, pages 201-219. Springer, London.
  62. Vecchiato, G., Astolfi, L., De Vico Fallani, F., Toppi, J., Aloise, F., Bez, F., Wei, D., Kong, W., Dai, J., Cincotti, F., Mattia, D., and Babiloni, F. (2011). On the use of EEG or MEG brain imaging tools in neuromarketing research. Comput Intell Neurosci, 2011:643489.
  63. Vi, C. and Subramanian, S. (2012). Detecting error-related negativity for interaction design. CHI 7812, page 493.
  64. Weber, J. (2007). Think Aloud Best Practices Study.
  65. Zander, T. O. and Kothe, C. (2011). Towards passive braincomputer interfaces: applying brain-computer interface technology to human-machine systems in general. J. Neural. Eng, 8(2):025005.
Download


Paper Citation


in Harvard Style

Frey J., Mühl C., Lotte F. and Hachet M. (2014). Review of the Use of Electroencephalography as an Evaluation Method for Human-Computer Interaction . In Proceedings of the International Conference on Physiological Computing Systems - Volume 1: PhyCS, ISBN 978-989-758-006-2, pages 214-223. DOI: 10.5220/0004708102140223


in Bibtex Style

@conference{phycs14,
author={Jérémy Frey and Christian Mühl and Fabien Lotte and Martin Hachet},
title={Review of the Use of Electroencephalography as an Evaluation Method for Human-Computer Interaction},
booktitle={Proceedings of the International Conference on Physiological Computing Systems - Volume 1: PhyCS,},
year={2014},
pages={214-223},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004708102140223},
isbn={978-989-758-006-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Physiological Computing Systems - Volume 1: PhyCS,
TI - Review of the Use of Electroencephalography as an Evaluation Method for Human-Computer Interaction
SN - 978-989-758-006-2
AU - Frey J.
AU - Mühl C.
AU - Lotte F.
AU - Hachet M.
PY - 2014
SP - 214
EP - 223
DO - 10.5220/0004708102140223