loading
Documents

Research.Publish.Connect.

Paper

Smart Like a Fox: How Clever Students Trick Dumb Automated Programming Assignment Assessment Systems

Topics: Blended Learning; Cloud-Based Learning and Assessment; Computer-Supported Collaborative Learning; eLearning Case Studies; e-Learning Hardware and Software; e-Learning Platforms, Portals; Intelligent Tutoring Systems; Learning Analytics; Massive Open Online Courses; Tools to Assess Learning; Virtual Labs and Virtual Classrooms

Author: Nane Kratzke

Affiliation: Lübeck University of Applied Sciences, Mönkhofer Weg 239, 23562 Lübeck and Germany

ISBN: 978-989-758-367-4

Keyword(s): Automated, Programming, Assignment, Assessment, Education, MOOC, Code injection, Moodle, VPL.

Abstract: This case study reports on two first-semester programming courses with more than 190 students. Both courses made use of automated assessments. We observed how students trick these systems by analysing the version history of suspect submissions. By analysing more than 3300 submissions, we revealed four astonishingly simple tricks (overfitting, evasion) and cheat-patterns (redirection, and injection) that students used to trick automated programming assignment assessment systems (APAAS). Although not the main focus of this study, it discusses and proposes corresponding counter-measures where appropriate. Nevertheless, the primary intent of this paper is to raise problem awareness and to identify and systematise observable problem patterns in a more formal approach. The identified immaturity of existing APAAS solutions might have implications for courses that rely deeply on automation like MOOCs. Therefore, we conclude to look at APAAS solutions much more from a security point of view (c ode injection). Moreover, we identify the need to evolve existing unit testing frameworks into more evaluation-oriented teaching solutions that provide better trick and cheat detection capabilities and differentiated grading support. (More)

PDF ImageFull Text

Download
Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 54.211.135.32

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Kratzke, N. (2019). Smart Like a Fox: How Clever Students Trick Dumb Automated Programming Assignment Assessment Systems.In Proceedings of the 11th International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-758-367-4, pages 15-26. DOI: 10.5220/0007424800150026

@conference{csedu19,
author={Nane Kratzke.},
title={Smart Like a Fox: How Clever Students Trick Dumb Automated Programming Assignment Assessment Systems},
booktitle={Proceedings of the 11th International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2019},
pages={15-26},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007424800150026},
isbn={978-989-758-367-4},
}

TY - CONF

JO - Proceedings of the 11th International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - Smart Like a Fox: How Clever Students Trick Dumb Automated Programming Assignment Assessment Systems
SN - 978-989-758-367-4
AU - Kratzke, N.
PY - 2019
SP - 15
EP - 26
DO - 10.5220/0007424800150026

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.