Evaluating a GPT-4 and Retrieval-Augmented Generation-Based Conversational Agent to Enhance Learning Experience in a MOOC

Fatma Miladi, Valéry Psyché, Awa Diattara, Nour El Mawas, Daniel Lemire

2025

Abstract

Massive Open Online Courses (MOOCs) face significant challenges due to low completion rates, primarily caused by insufficient personalized support for learners. To address this, we developed a pedagogical AI-powered conversational agent enhanced with Retrieval-Augmented Generation (RAG) to provide real-time, contextually relevant support. Our evaluation with 25 learners demonstrated a statistically significant knowledge gain in the experimental group compared to the control group. Additionally, the agent achieved a high System Usability Scale (SUS) score. These findings highlight the potential of AI technologies to enhance online learning environments and inform future research on their role as learning companions in distance education.

Paper Citation

in Harvard Style

Miladi F., Psyché V., Diattara A., El Mawas N. and Lemire D. (2025). Evaluating a GPT-4 and Retrieval-Augmented Generation-Based Conversational Agent to Enhance Learning Experience in a MOOC. In Proceedings of the 17th International Conference on Computer Supported Education - Volume 2: CSEDU; ISBN 978-989-758-746-7, SciTePress, pages 347-354. DOI: 10.5220/0013366100003932

in Bibtex Style

@conference{csedu25,
author={Fatma Miladi and Valéry Psyché and Awa Diattara and Nour El Mawas and Daniel Lemire},
title={Evaluating a GPT-4 and Retrieval-Augmented Generation-Based Conversational Agent to Enhance Learning Experience in a MOOC},
booktitle={Proceedings of the 17th International Conference on Computer Supported Education - Volume 2: CSEDU},
year={2025},
pages={347-354},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013366100003932},
isbn={978-989-758-746-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Computer Supported Education - Volume 2: CSEDU
TI - Evaluating a GPT-4 and Retrieval-Augmented Generation-Based Conversational Agent to Enhance Learning Experience in a MOOC
SN - 978-989-758-746-7
AU - Miladi F.
AU - Psyché V.
AU - Diattara A.
AU - El Mawas N.
AU - Lemire D.
PY - 2025
SP - 347
EP - 354
DO - 10.5220/0013366100003932
PB - SciTePress