Where Is the Evidence? A Plugin for Auditing Moodle’s Learning Analytics

Linda Fernsel, Yannick Kalff, Katharina Simbeck

2024

Abstract

The paper presents the work-in-progress development of a Moodle plugin to improve the auditability of Moodle’s Learning Analytics component. Future legislation, such as the EU AI Act, will require audits and “conformity assessments” of “high-risk” AI systems. Educational applications can be considered high-risk systems due to their important role in individual life and career paths. Therefore, their correctness, fairness, and efficiency must be assessed. However, auditing of the Learning Analytics functions in Moodle is limited. No suitable test-data is available, models and configurations are not persistent and only aggregated quality metrics are returned that are insufficient to assess fairness. The plugin addresses these issues and provides a data interface to extract data for audits. The plugin allows to a) upload and select data for the audit, b) clearly differentiate between model configuration and trained models, c) keep trained models, their configuration and underlying data for future inspections and comparisons, and finally, d) the plugin saves raw predictions for further analysis. The plugin enables the audit of Moodle’s Learning Analytics and its underlying AI models and contributes to increased fairness and trustworthiness of Learning Analytics as well as its legally compliant application.

Paper Citation

in Harvard Style

Fernsel L., Kalff Y. and Simbeck K. (2024). Where Is the Evidence? A Plugin for Auditing Moodle’s Learning Analytics. In Proceedings of the 16th International Conference on Computer Supported Education - Volume 2: CSEDU; ISBN 978-989-758-697-2, SciTePress, pages 262-269. DOI: 10.5220/0012689800003693

in Bibtex Style

@conference{csedu24,
author={Linda Fernsel and Yannick Kalff and Katharina Simbeck},
title={Where Is the Evidence? A Plugin for Auditing Moodle’s Learning Analytics},
booktitle={Proceedings of the 16th International Conference on Computer Supported Education - Volume 2: CSEDU},
year={2024},
pages={262-269},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012689800003693},
isbn={978-989-758-697-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Conference on Computer Supported Education - Volume 2: CSEDU
TI - Where Is the Evidence? A Plugin for Auditing Moodle’s Learning Analytics
SN - 978-989-758-697-2
AU - Fernsel L.
AU - Kalff Y.
AU - Simbeck K.
PY - 2024
SP - 262
EP - 269
DO - 10.5220/0012689800003693
PB - SciTePress