How Far Can We Trust the Predictions of Learning Analytics Systems?

Amal Ben Soussia, Anne Boyer

2023

Abstract

Prediction systems based on Machine Learning (ML) models for teachers are widely used in the Learning Analytics (LA) field to address the problem of high failure rates in online learning. One objective of these systems is to identify at-risk of failure learners so that teachers can intervene effectively with them. Therefore, teachers’ trust in the reliability of the predictive performance of these systems is of great importance. However, despite the relevance of this notion of trust, the literature does not propose particular methods to measure the trust to be granted to the system results. In this paper, we develop an approach to measure a teacher’s trust in the prediction accuracy of an LA system. For this aim, we define three trust granularities, including: the overall trust, trust per class label and trust per prediction. For each trust granularity, we proceed to the calculation of a Trust Index (TI) using the concepts of confidence level and confidence interval of statistics. As a proof of concept, we apply this approach on a system using the Random Forest (RF) model and real data of online k-12 learners.

Paper Citation

in Harvard Style

Ben Soussia A. and Boyer A. (2023). How Far Can We Trust the Predictions of Learning Analytics Systems?. In Proceedings of the 15th International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-758-641-5, SciTePress, pages 150-157. DOI: 10.5220/0012057800003470

in Bibtex Style

@conference{csedu23,
author={Amal Ben Soussia and Anne Boyer},
title={How Far Can We Trust the Predictions of Learning Analytics Systems?},
booktitle={Proceedings of the 15th International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2023},
pages={150-157},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012057800003470},
isbn={978-989-758-641-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - How Far Can We Trust the Predictions of Learning Analytics Systems?
SN - 978-989-758-641-5
AU - Ben Soussia A.
AU - Boyer A.
PY - 2023
SP - 150
EP - 157
DO - 10.5220/0012057800003470
PB - SciTePress