Fine-Grained Prediction of Cognitive Workload in a Modern Working Environment by Utilizing Short-Term Physiological Parameters

Timm Hörmann, Marc Hesse, Peter Christ, Michael Adams, Christian Menßen, Ulrich Rückert

2016

Abstract

In this paper we present a method to predict cognitive workload during the interaction with a tablet computer. To set up a predictor that estimates the reflected self-reported cognitive workload we analyzed the information gain of heart rate, electrodermal activity and user input (touch) based features. From the derived optimal feature set we present a Gaussian Process based learner that enables fine-grained and short term detection of cognitive workload. Average inter-subject accuracy in 10-fold cross validation is 74.1 % for the fine-grained 5-class problem and 96.0 % for the binary class problem.

References

  1. Bishop, C. M. (2006). Pattern Recognition and Machine Learning (Information Science and Statistics), chapter Kernel Methods, pages 291-323. Springer New York.
  2. Botthof, A. and Hartmann, E. (2015). Zukunft der Arbeit in Industrie 4.0 - Neue Perspektiven und offene Fragen. In Botthof, A. and Hartmann, E. A., editors, Zukunft der Arbeit in Industrie 4.0, pages 161-163. Springer Berlin Heidelberg.
  3. Bowling, N. A. and Kirkendall, C. (2012). Workload: A Review of Causes, Consequences, and Potential Interventions, pages 221-238. John Wiley & Sons, Ltd.
  4. Cain, B. (2007). A review of the mental workload literature. In RTO-TR-HFM-121-Part-II. NATO Science and Technology Organization.
  5. Choi, J., Ahmed, B., and Gutierrez-Osuna, R. (2012). Development and evaluation of an ambulatory stress monitor based on wearable sensors. IEEE Transactions on Information Technology in Biomedicine, 16(2):279-286.
  6. Choi, J. and Gutierrez-Osuna, R. (2009). Using heart rate monitors to detect mental stress. In Sixth International Workshop on Wearable and Implantable Body Sensor Networks, pages 219-223.
  7. Dietterich, T. G. and Bakiri, G. (1995). Solving multiclass learning problems via error-correcting output codes. Journal of artificial intelligence research, pages 263- 286.
  8. eSense Skin Response (2015). Biofeedback system. Mindfield Biosystems Ltd., Berlin, Germany.
  9. Fürnkranz, J. (2010). Decision tree. In Sammut, C. and Webb, G., editors, Encyclopedia of Machine Learning, pages 263-267. Springer US.
  10. Harris, W., Hancock, P., and Arthur, E. (1993). The effect of taskload projection on automation use, performance, and workload. In Proceedings of the Seventh International Symposium on Aviation Psychology.
  11. Hart, S. G. and Staveland, L. E. (1988). Development of NASA-TLX (task load index): Results of empirical and theoretical research. In Human Mental Workload, volume 52, pages 139-183. North-Holland.
  12. Healey, J. and Picard, R. (2005). Detecting Stress During Real-World Driving Tasks Using Physiological Sensors. IEEE Transactions on Intelligent Transportation Systems, 6(2):156-166.
  13. Isshiki, H. and Yamamoto, Y. (1994). Instrument for monitoring arousal level using electrodermal activity. In Proceedings of IEEE International Conference on Instrumentation and Measurement Technology, pages 975-978. IEEE.
  14. Jorna, P. G. (1992). Spectral analysis of heart rate and psychological state: a review of its validity as a workload index. Biological psychology, 34(2-3):237-257.
  15. Karthikeyan, P., Murugappan, M., and Yaacob, S. (2013). Detection of human stress using short-term ECG and HRV signals. Journal of Mechanics in Medicine and Biology, 13(02):1350038.
  16. Keogh, E. (2010). Nearest neighbor. In Sammut, C. and Webb, G., editors, Encyclopedia of Machine Learning, pages 714-715. Springer US.
  17. Malik, M., Bigger, J. T., Camm, A. J., Kleiger, R. E., Malliani, A., Moss, A. J., and Schwartz, P. J. (1996). Heart rate variability. European Heart Journal, 17(3):354-381.
  18. MATLAB (2015). Version 8.5.0 (R2015a). The MathWorks Inc., Natick, Massachusetts.
  19. MIO Alpha (2013). Heart rate watch. Physical Enterprises Inc. (Mio Global), Canada, Vancouver.
  20. Nexus 10 (2012). GT-P8110. Google Inc.; Samsung Electronics.
  21. Polar H6 (2012). Model: X9. Polar Electro Oy, Finland, Kempele.
  22. Quadrianto, N., Kersting, K., and Xu, Z. (2010). Gaussian process. In Sammut, C. and Webb, G., editors, Encyclopedia of Machine Learning, pages 428-439. Springer US.
  23. QuickAmp (2015). Biofeedbacksystem. Brain Products GmbH, Gilching, Germany.
  24. Rasmussen, C. E. and Nickisch, H. (2010). Gaussian processes for machine learning (GPML) toolbox. Journal of Machine Learning Research, 11:3011-3015.
  25. Rouse, W., Edwards, S., and Hammer, J. M. (1993). Modeling the dynamics of mental workload and human performance in complex systems. IEEE Transactions on Systems, Man, and Cybernetics, 23(6):1662-1671.
  26. Singh, D., Vinod, K., and Saxena, S. (2004). Sampling frequency of the RR interval time series for spectral analysis of heart rate variability. Journal of medical engineering & technology, 28(6):263-272.
  27. Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6):643-662.
  28. Sun, F.-T., Kuo, C., Cheng, H.-T., Buthpitiya, S., Collins, P., and Griss, M. (2012). Activity-aware mental stress detection using physiological sensors. In Gris, M. and Yang, G., editors, Mobile Computing, Applications, and Services, volume 76, pages 211-230. Springer Berlin Heidelberg.
  29. Tarvainen, M., Ranta-aho, P., and Karjalainen, P. (2002). An advanced detrending method with application to HRV analysis. IEEE Transactions on Biomedical Engineering, 49(2):172-175.
  30. Wallhoff, F., Ablassmeier, M., Bannat, A., Buchta, S., Rauschert, A., Rigoll, G., and Wiesbeck, M. (2007). Adaptive human-machine interfaces in cognitive production environments. In Proceedings of IEEE International Conference on Multimedia and Expo, pages 2246-2249.
  31. Webb, G. (2010). Naïve bayes. In Sammut, C. and Webb, G., editors, Encyclopedia of Machine Learning, pages 713-714. Springer US.
  32. Wijsman, J., Grundlehner, B., Liu, H., Hermens, H., and Penders, J. (2011). Towards mental stress detection using wearable physiological sensors. In Proceedings of IEEE Engineering in Medicine and Biology Society, pages 1798-1801.
  33. Witten, I. H. and Frank, E. (2005). Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, second edition.
  34. Young, M. S. and Stanton, N. A. (2002). Attention and automation: New perspectives on mental underload and performance. Theoretical Issues in Ergonomics Science, 3(2):178-194.
Download


Paper Citation


in Harvard Style

Hörmann T., Hesse M., Christ P., Adams M., Menßen C. and Rückert U. (2016). Fine-Grained Prediction of Cognitive Workload in a Modern Working Environment by Utilizing Short-Term Physiological Parameters . In Proceedings of the 9th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 4: BIOSIGNALS, (BIOSTEC 2016) ISBN 978-989-758-170-0, pages 42-51. DOI: 10.5220/0005665000420051


in Bibtex Style

@conference{biosignals16,
author={Timm Hörmann and Marc Hesse and Peter Christ and Michael Adams and Christian Menßen and Ulrich Rückert},
title={Fine-Grained Prediction of Cognitive Workload in a Modern Working Environment by Utilizing Short-Term Physiological Parameters},
booktitle={Proceedings of the 9th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 4: BIOSIGNALS, (BIOSTEC 2016)},
year={2016},
pages={42-51},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005665000420051},
isbn={978-989-758-170-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 4: BIOSIGNALS, (BIOSTEC 2016)
TI - Fine-Grained Prediction of Cognitive Workload in a Modern Working Environment by Utilizing Short-Term Physiological Parameters
SN - 978-989-758-170-0
AU - Hörmann T.
AU - Hesse M.
AU - Christ P.
AU - Adams M.
AU - Menßen C.
AU - Rückert U.
PY - 2016
SP - 42
EP - 51
DO - 10.5220/0005665000420051