Assessment of Training Progression on a Surgical Simulator Using Machine Learning and Explainable Artificial Intelligence Techniques

Constantinos Loukas, Konstantina Prevezanou

2025

Abstract

Surgical training on VR simulators provides an efficient education paradigm in laparoscopic surgery. Most methods for skills assessment focus on the analysis of video and kinematic data for self-proclaimed skill classification and technical score prediction. In this paper we evaluate a machine learning (ML) framework for classifying the trainee’s performance with respect to the phase of training progression (beginning vs. end of training and beginning vs. middle vs. end of training). In addition, we leverage techniques from the field of Explainable Artificial Intelligence (XAI) to obtain interpretations on the employed black-box ML classifiers. Three surgical training tasks with significant educational value were selected from a training curriculum followed by 23 medical students. Five machine learning algorithms and two model-agnostic XAI methods were evaluated using performance metrics generated by the simulator during task performance. For all surgical tasks, the accuracy was >84% and >86% in the 2- and 3-class classification experiments, respectively. The XAI methods seem to agree on the relative impact of each performance metric. Features related to hand-eye coordination and bimanual dexterity (e.g. economy of movements, instrument pathlength and number of movements), play the most important role in explaining the classification results.

Paper Citation

in Harvard Style

Loukas C. and Prevezanou K. (2025). Assessment of Training Progression on a Surgical Simulator Using Machine Learning and Explainable Artificial Intelligence Techniques. In Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM; ISBN 978-989-758-730-6, SciTePress, pages 465-474. DOI: 10.5220/0013109500003905

in Bibtex Style

@conference{icpram25,
author={Constantinos Loukas and Konstantina Prevezanou},
title={Assessment of Training Progression on a Surgical Simulator Using Machine Learning and Explainable Artificial Intelligence Techniques},
booktitle={Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM},
year={2025},
pages={465-474},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013109500003905},
isbn={978-989-758-730-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM
TI - Assessment of Training Progression on a Surgical Simulator Using Machine Learning and Explainable Artificial Intelligence Techniques
SN - 978-989-758-730-6
AU - Loukas C.
AU - Prevezanou K.
PY - 2025
SP - 465
EP - 474
DO - 10.5220/0013109500003905
PB - SciTePress