A Systematic Assessment of Operational Metrics for Modeling Operator Functional State

Jean-François Gagnon, Olivier Gagnon, Daniel Lafond, Mark Parent, Sébastien Tremblay

2016

Abstract

This paper addresses critical issues and reports key findings with regards to the development of participant-generic operator functional state (OFS) models in the context of cognitive work. Conceptually, this research is concerned with the nature of the relationship between the physiological state of individuals and human performance. Participants were physiologically monitored (cardiac, respiratory, and eye activity) during the execution a set of two cognitive tasks – n-back and visual search – for which there were two levels of difficulty. Levels of difficulty were associated with levels of mental workload. Performance on the tasks was also monitored and linked with OFS. Modeling of the relationship between physiological state and OFS involved systematic manipulation of three parameters: (1) size of smoothing window for performance, (2) performance decrement threshold for labelling functional and sub-functional states, and (3) the mode of classification being either prospective or descriptive. Modeling was performed using two types of classifiers. Results show that (1) models that use bio-behavioral data were capable of classifying performance on new participant data above chance, (2) levels of mental workload were better classified than OFS, (3) size of smoothing window had a significant impact on classifier performance, and (4) size of smoothing window, threshold values, and classifier type had a significant impact on sensitivity and specificity. Implications for the use of OFS models in operational contexts are discussed.

References

  1. Boonnithi, S., Phongsuphap, S., 2011. Comparison of heart rate variability measures for mental stress detection, in: Computing in Cardiology, 2011. pp. 85- 88.
  2. Bracken, B.K., Palmon, N., Romero, V., Pfautz, J., Cooke, N.J., 2014. A Prototype Toolkit for Sensing and Modeling Individual and Team State. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 58, 949-953. doi:10.1177/154193121458 1199.
  3. Brouwer, A.-M., Zander, T.O., van Erp, J.B.F., Korteling, J.E., Bronkhorst, A.W., 2015. Using neurophysiological signals that reflect cognitive or affective state: six recommendations to avoid common pitfalls. Frontiers in Neuroscience 9. doi:10.3389/fnins.2015.00136.
  4. Carter, R., Cheuvront, S. N., and Sawka, M. N., 2004. Operator Functional State Assessment (l'évaluation de l'aptitude opérationnelle de l'opérateur humain). Army research institute.
  5. Dirican, A.C., Göktürk, M., 2011. Psychophysiological measures of human cognitive states applied in human computer interaction. Procedia Computer Science, World Conference on Information Technology 3, 1361-1367. doi:10.1016/j.procs.2011.01.016.
  6. Durantin, G., Gagnon, J.-F., Tremblay, S., Dehais, F., 2014. Using near infrared spectroscopy and heart rate variability to detect mental overload. Behavioural Brain Research 259, 16-23. doi:10.1016/j.bbr.20 13.10.042.
  7. Durkee, K.T., Pappada, S.M., Ortiz, A.E., Feeney, J.J., Galster, S.M., 2015. System Decision Framework for Augmenting Human Performance Using Real-Time Workload Classifiers. Presented at the 2015 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), Orlando, FL.
  8. Eggemeier, F.T., Wilson, G.F., Kramer, A.F., Damos, D.L., 1991. Workload assessment in multi-task environments. Multiple-task performance 207-216.
  9. Gagnon, O., Lafond, D., Gagnon, J-F., and Parizeau, M., 2016. Comparing Methods for Assessing Operator Functional State. Proceedings of the 2016 IEEE International Inter-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), San Diego, CA, USA, March 21-25.
  10. Gaillard, A.W., 2003. Fatigue assessment and performance protection, NATO Science Series Sub Series I Life and Behavioural Sciences 355, 24-35,.
  11. Hogervorst, M.A., Brouwer, A.-M., van Erp, J.B.F., 2014. Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload. Frontiers in Neuroscience 8. doi:10.3389/fnins.2014.00322.
  12. Kuhn, M., 2015. caret: Classification and regression training. Astrophysics Source Code Library 1, 05003.
  13. Matthews, G., Reinerman-Jones, L.E., Barber, D.J., Abich, J., 2015. The Psychometrics of Mental Workload Multiple Measures Are Sensitive but Divergent. Human Factors: The Journal of the Human Factors and Ergonomics Society 57, 125-143. doi:10.1177/0018720814539505.
  14. Nourbakhsh, N., Wang, Y., and Chen, F., 2013. GSR and blink features for cognitive load classification. In Human-Computer Interaction-INTERACT, pp. 159- 166. Springer Berlin Heidelberg.
  15. Overbeek, T. J., van Boxtel, A., and Westerink, J. H., 2014. Respiratory sinus arrhythmia responses to cognitive tasks: Effects of task factors and RSA indices. Biological psychology 99, 1-14.
  16. Régis, N., Dehais, F., Rachelson, E., Thooris, C., Pizziol, S., Causse, M., and Tessier, C., 2014. Formal Detection of Attentional Tunneling in Human Operator-Automation Interactions. IEEE Transactions on Human-Machine Systems 44(3), 326-336.
  17. Rodríguez-Liñares, L., Vila, X., Mendez, A., Lado, M., and Olivieri, D., 2008. RHRV: An R-based software package for heart rate variability analysis of ECG recordings. In 3rd Iberian Conference in Systems and Information Technologies (CISTI 2008), Vigo, Spain.
  18. Tobon D.V., Falk, T., and Maier, M., 2014. MS-QI: A Modulation Spectrum-Based ECG Quality Index for Telehealth Applications. IEEE Transactions on Biomedical Engineering. 99, pp. 1-1.
  19. Torgo, L. 2010. Data Mining with R, learning with case studies Chapman and Hall/CRC. URL: http://www.dcc.fc.up.pt/ltorgo/DataMiningWithR.
  20. Wang, Z., Hope, R.M., Wang, Z., Ji, Q., Gray, W.D., 2012. Cross-subject workload classification with a hierarchical Bayes model. NeuroImage, Neuroergonomics: The human brain in action and at work 59, 64-69. doi:10.1016/j.neuroimage.2011.0 7.094.
  21. Wijsman, J., Grundlehner, B., Liu, H., Penders, J., Hermens, H., 2013. Wearable Physiological Sensors Reflect Mental Stress State in Office-Like Situations, in: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII). pp. 600-605. doi:10.1109/ACII.2013.105.
  22. Williamon, A., Aufegger, L., Wasley, D., Looney, D., Mandic, D.P., 2013. Complexity of physiological responses decreases in high-stress musical performance. Journal of The Royal Society Interface 10. doi:10.1098/rsif.2013.0719.
  23. Wilson, G. F.and Russell, C. A., 2003. Real-Time Assessment of Mental Workload Using Psychophysiological Measures and Artificial Neural Networks. Human Factors, 45, 635-643.
  24. Yin, Z., Zhang, J., 2014. Operator functional state classification using least-square support vector machine based recursive feature elimination technique. Computer Methods and Programs in Biomedicine 113, 101-115. doi:10.1016/j.cmpb.2013.09.007.
Download


Paper Citation


in Harvard Style

Gagnon J., Gagnon O., Lafond D., Parent M. and Tremblay S. (2016). A Systematic Assessment of Operational Metrics for Modeling Operator Functional State . In Proceedings of the 3rd International Conference on Physiological Computing Systems - Volume 1: PhyCS, ISBN 978-989-758-197-7, pages 15-23. DOI: 10.5220/0005921600150023


in Bibtex Style

@conference{phycs16,
author={Jean-François Gagnon and Olivier Gagnon and Daniel Lafond and Mark Parent and Sébastien Tremblay},
title={A Systematic Assessment of Operational Metrics for Modeling Operator Functional State},
booktitle={Proceedings of the 3rd International Conference on Physiological Computing Systems - Volume 1: PhyCS,},
year={2016},
pages={15-23},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005921600150023},
isbn={978-989-758-197-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 3rd International Conference on Physiological Computing Systems - Volume 1: PhyCS,
TI - A Systematic Assessment of Operational Metrics for Modeling Operator Functional State
SN - 978-989-758-197-7
AU - Gagnon J.
AU - Gagnon O.
AU - Lafond D.
AU - Parent M.
AU - Tremblay S.
PY - 2016
SP - 15
EP - 23
DO - 10.5220/0005921600150023