Measuring the Effect of Classification Accuracy on User Experience in a Physiological Game

Gregor Geršak, Sean M. McCrea, Domen Novak

2016

Abstract

Physiological games use classification algorithms to extract information about the player from physiological measurements and adapt game difficulty accordingly. However, little is known about how the classification accuracy affects the overall user experience and how to measure this effect. Following up on a previous study, we artificially predefined classification accuracy in a game of Snake where difficulty increases or decreases after each round. The game was played in a laboratory setting by 110 participants at different classification accuracies. The participants reported their satisfaction with the difficulty adaptation algorithm as well as their in-game fun, with 85 participants using electronic questionnaires and 25 using paper questionnaires. We observed that the classification accuracy must be at least 80% for the physiological game to be accepted by users and that there are notable differences between different methods of measuring the effect of classification accuracy. The results also show that laboratory settings are more effective than online settings, and paper questionnaires exhibit higher correlations between classification accuracy and user experience than electronic questionnaires. Implications for the design and evaluation of physiological games are presented.

References

  1. Chanel, G. et al., 2011. Emotion assessment from physiological signals for adaptation of game difficulty. IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans, 41(6), pp.1052-1063.
  2. Fairclough, S.H., 2009. Fundamentals of physiological computing. Interacting with Computers, 21(1-2), pp.133-145.
  3. Gilleade, K., Dix, A. and Allanson, J., 2005. Affective videogames and modes of affective gaming: assist me, challenge me, emote me. In Proceedings of DiGRA 2005.
  4. Koenig, A. et al., 2011. Real-time closed-loop control of cognitive load in neurological patients during robotassisted gait training. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 19(4), pp.453-64.
  5. van de Laar, B. et al., 2013. How much control is enough? Influence of unreliable input on user experience. IEEE Transactions on Cybernetics, 43(6), pp.1584-1592.
  6. Liu, C. et al., 2009. Dynamic difficulty adjustment in computer games through real-time anxiety-based affective feedback. International Journal of HumanComputer Interaction, 25(6), pp.506-529.
  7. Liu, C. et al., 2008. Online affect detection and robot behavior adaptation for intervention of children with autism. IEEE Transactions on Robotics, 24(4), pp.883-896.
  8. McAuley, E., Duncan, T. and Tammen, V. V., 1989. Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: a confirmatory factor analysis. Research Quarterly for Exercise and Sport, 60(1), pp.48-58.
  9. Novak, D. et al., 2011. Psychophysiological measurements in a biocooperative feedback loop for upper extremity rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 19(4), pp.400-410.
  10. Novak, D., Mihelj, M. and Munih, M., 2012. A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing. Interacting with Computers, 24, pp.154- 172.
  11. Novak, D., Nagle, A. and Riener, R., 2014. Linking recognition accuracy and user experience in an affective feedback loop. IEEE Transactions on Affective Computing, 5(2), pp.168-172.
  12. Oppenheimer, D.M., Meyvis, T. and Davidenko, N., 2009. Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45(4), pp.867-872.
  13. Paolacci, G., Chandler, J. and Ipeirotis, P.G., 2010. Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5(5), pp.411-419.
  14. Shirzad, N. and Van der Loos, H.F.M., 2016. Evaluating the user experience of exercising reaching motions with a robot that predicts desired movement difficulty. Journal of Motor Behavior, 48(1), pp.31-46.
Download


Paper Citation


in Harvard Style

Geršak G., McCrea S. and Novak D. (2016). Measuring the Effect of Classification Accuracy on User Experience in a Physiological Game . In Proceedings of the 3rd International Conference on Physiological Computing Systems - Volume 1: PhyCS, ISBN 978-989-758-197-7, pages 80-87. DOI: 10.5220/0005940300800087


in Bibtex Style

@conference{phycs16,
author={Gregor Geršak and Sean M. McCrea and Domen Novak},
title={Measuring the Effect of Classification Accuracy on User Experience in a Physiological Game},
booktitle={Proceedings of the 3rd International Conference on Physiological Computing Systems - Volume 1: PhyCS,},
year={2016},
pages={80-87},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005940300800087},
isbn={978-989-758-197-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Physiological Computing Systems - Volume 1: PhyCS,
TI - Measuring the Effect of Classification Accuracy on User Experience in a Physiological Game
SN - 978-989-758-197-7
AU - Geršak G.
AU - McCrea S.
AU - Novak D.
PY - 2016
SP - 80
EP - 87
DO - 10.5220/0005940300800087