Auto-Grader Feedback Utilization and Its Impacts: An Observational Study Across Five Community Colleges

Adam Zhang, Heather Burte, Jaromir Savelka, Christopher Bogart, Majd Sakr

2025

Abstract

Automated grading systems, or auto-graders, have become ubiquitous in programming education, and the way they generate feedback has become increasingly automated as well. However, there is insufficient evidence regarding auto-grader feedback’s effectiveness in improving student learning outcomes, in a way that differentiates students who utilized the feedback and students who did not. In this study, we fill this critical gap. Specifically, we analyze students’ interactions with auto-graders in an introductory Python programming course, offered at five community colleges in the United States. Our results show that students checking the feedback more frequently tend to get higher scores from their programming assignments overall. Our results also show that a submission that follows a student checking the feedback tends to receive a higher score than a submission that follows a student ignoring the feedback. Our results provide evidence on auto-grader feedback’s effectiveness, encourage their increased utilization, and call for future work to continue their evaluation in this age of automation.

Paper Citation

in Harvard Style

Zhang A., Burte H., Savelka J., Bogart C. and Sakr M. (2025). Auto-Grader Feedback Utilization and Its Impacts: An Observational Study Across Five Community Colleges. In Proceedings of the 17th International Conference on Computer Supported Education - Volume 1: CSEDU; ISBN 978-989-758-746-7, SciTePress, pages 356-363. DOI: 10.5220/0013276800003932

in Bibtex Style

@conference{csedu25,
author={Adam Zhang and Heather Burte and Jaromir Savelka and Christopher Bogart and Majd Sakr},
title={Auto-Grader Feedback Utilization and Its Impacts: An Observational Study Across Five Community Colleges},
booktitle={Proceedings of the 17th International Conference on Computer Supported Education - Volume 1: CSEDU},
year={2025},
pages={356-363},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013276800003932},
isbn={978-989-758-746-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Computer Supported Education - Volume 1: CSEDU
TI - Auto-Grader Feedback Utilization and Its Impacts: An Observational Study Across Five Community Colleges
SN - 978-989-758-746-7
AU - Zhang A.
AU - Burte H.
AU - Savelka J.
AU - Bogart C.
AU - Sakr M.
PY - 2025
SP - 356
EP - 363
DO - 10.5220/0013276800003932
PB - SciTePress