Introducing a Framework for Code based Fairness Audits of Learning Analytics Systems on the Example of Moodle Learning Analytics

Hassan Tagharobi, Katharina Simbeck

2022

Abstract

Machine learning based predictive systems are increasingly used in various areas, including learning analytics (LA) systems. LA systems provide educators with an analysis of students’ progress and offer predictions about their success. Although predictive systems provide new opportunities and convenience, studies show that they harbor risks for biased or even discriminatory outcomes. To detect and solve these discriminatory issues and examine algorithmic fairness, different approaches have been introduced. The majority of purposed approaches study the behavior of predictive systems using sample data. However, if the source code is available, e.g., for open-source projects, auditing it can further improve the examination of algorithmic fairness. In this paper, we introduce a framework for an independent audit of algorithmic fairness using all publicly available resources. We applied our framework on Moodle learning analytics and examined its fairness for a defined set of criteria. Our fairness audit shows that Moodle doesn’t use protected attributes, e.g., gender, ethnicity, in its predictive process. However, we detected some issues in data distribution and processing, which could potentially affect the fairness of the system. Furthermore, we believe that the system should provide users with more detailed evaluation metrics to enable proper assessment of the quality of learning analytics models.

Paper Citation

in Harvard Style

Tagharobi H. and Simbeck K. (2022). Introducing a Framework for Code based Fairness Audits of Learning Analytics Systems on the Example of Moodle Learning Analytics. In Proceedings of the 14th International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-758-562-3, pages 45-55. DOI: 10.5220/0010998900003182

in Bibtex Style

@conference{csedu22,
author={Hassan Tagharobi and Katharina Simbeck},
title={Introducing a Framework for Code based Fairness Audits of Learning Analytics Systems on the Example of Moodle Learning Analytics},
booktitle={Proceedings of the 14th International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2022},
pages={45-55},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010998900003182},
isbn={978-989-758-562-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - Introducing a Framework for Code based Fairness Audits of Learning Analytics Systems on the Example of Moodle Learning Analytics
SN - 978-989-758-562-3
AU - Tagharobi H.
AU - Simbeck K.
PY - 2022
SP - 45
EP - 55
DO - 10.5220/0010998900003182