Audits for Trust: An Auditability Framework for AI-Based Learning Analytics Systems

Linda Fernsel, Yannick Kalff, Katharina Simbeck

2025

Abstract

Audits contribute to the trustworthiness of Learning Analytics (LA) systems that integrate Artificial Intelligence (AI) and may be legally required in the future. We argue that the efficacy of an audit depends on the auditability of the audited system. Therefore, systems need to be designed with auditability in mind. We present a framework for assessing the auditability of AI-integrating systems in education that consists of three parts: (1) verifiable claims about the validity, utility and ethics of the system, (2) evidence on subjects (data, models, or the system) in different types (documentation, raw sources and logs) to back or refute claims, (3) means to validate evidence such as technical APIs, monitoring tools, or explainable AI principles must be accessible to auditors. We apply the framework to assess the auditability of the Learning Management System Moodle, which supports an AI-integrating dropout prediction system. Moodle’s auditability is limited by incomplete documentation, insufficient monitoring capabilities, and a lack of available test data.

Paper Citation

in Harvard Style

Fernsel L., Kalff Y. and Simbeck K. (2025). Audits for Trust: An Auditability Framework for AI-Based Learning Analytics Systems. In Proceedings of the 17th International Conference on Computer Supported Education - Volume 2: CSEDU; ISBN 978-989-758-746-7, SciTePress, pages 51-62. DOI: 10.5220/0013254300003932

in Bibtex Style

@conference{csedu25,
author={Linda Fernsel and Yannick Kalff and Katharina Simbeck},
title={Audits for Trust: An Auditability Framework for AI-Based Learning Analytics Systems},
booktitle={Proceedings of the 17th International Conference on Computer Supported Education - Volume 2: CSEDU},
year={2025},
pages={51-62},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013254300003932},
isbn={978-989-758-746-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Computer Supported Education - Volume 2: CSEDU
TI - Audits for Trust: An Auditability Framework for AI-Based Learning Analytics Systems
SN - 978-989-758-746-7
AU - Fernsel L.
AU - Kalff Y.
AU - Simbeck K.
PY - 2025
SP - 51
EP - 62
DO - 10.5220/0013254300003932
PB - SciTePress