Leveraging NLP and Machine Learning for English (L1) Writing Assessment in Developmental Education

Miguel Da Corte, Miguel Da Corte, Jorge Baptista, Jorge Baptista

2024

Abstract

This study investigates using machine learning and linguistic features to predict placements in Developmental Education (DevEd) courses based on English (L1) writing proficiency. Placement in these courses is often performed using systems like ACCUPLACER, which automatically assesses and scores standardized writing assignments in entrance exams. Literature on ACCUPLACER’s assessment methods and the features accounted for in the scoring process is scarce. To identify the linguistic features important for placement decisions, 100 essays were randomly selected and analyzed from a pool of essays written by 290 native speakers. A total of 457 Linguistic attributes were extracted using COH-METRIX (106), the Common Text Analysis Platform (CTAP) (330), plus 21 DevEd-specific features produced by the manual annotation of the corpus. Using the ORANGE Text Mining toolkit, several supervised Machine-learning (ML) experiments with two classification scenarios (full and split sample essays) were conducted to determine the best linguistic features and best-performing ML algorithm. Results revealed that the Naive Bayes, with a selection of the 30 highest-ranking features (21 CTAP, 7 COH-METRIX, 2 DevEd-specific) based on the Information Gain scoring method, achieved a classification accuracy (CA) of 77.3%, improving to 81.8% with 60 features. This approach surpassed the baseline accuracy of 72.7% for the full essay scenario, demonstrating enhanced placement accuracy and providing new insights into students’ linguistic skills in DevEd.

Paper Citation

in Harvard Style

Da Corte M. and Baptista J. (2024). Leveraging NLP and Machine Learning for English (L1) Writing Assessment in Developmental Education. In Proceedings of the 16th International Conference on Computer Supported Education - Volume 2: CSEDU; ISBN 978-989-758-697-2, SciTePress, pages 128-140. DOI: 10.5220/0012740500003693

in Bibtex Style

@conference{csedu24,
author={Miguel Da Corte and Jorge Baptista},
title={Leveraging NLP and Machine Learning for English (L1) Writing Assessment in Developmental Education},
booktitle={Proceedings of the 16th International Conference on Computer Supported Education - Volume 2: CSEDU},
year={2024},
pages={128-140},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012740500003693},
isbn={978-989-758-697-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Conference on Computer Supported Education - Volume 2: CSEDU
TI - Leveraging NLP and Machine Learning for English (L1) Writing Assessment in Developmental Education
SN - 978-989-758-697-2
AU - Da Corte M.
AU - Baptista J.
PY - 2024
SP - 128
EP - 140
DO - 10.5220/0012740500003693
PB - SciTePress