Using BERT and XLNET for the Automatic Short Answer Grading Task

Hadi Ghavidel, Amal Zouaq, Michel Desmarais

2020

Abstract

Over the last decade, there has been a considerable amount of research in automatic short answer grading (ASAG). The majority of previous experiments were based on a feature engineering approach and used manually-engineered statistical, lexical, grammatical and semantic features for ASAG. In this study, we aim for an approach that is free from manually-engineered features and propose an architecture for deep learning based on the newly-introduced BERT (Bidirectional Encoder Representations from Transformers) and XLNET (Extra Long Network) classifiers. We report the results achieved over one of the most popular dataset for ASAG, SciEntBank. Compared to past works for the SemEval-2013 2-way, 3-way and 5-way tasks, we obtained better or competitive performance with BERT Base (cased and uncased) and XLNET Base (cased) using a reference-based approach (considering students and model answers) and without any type of hand-crafted features.

Paper Citation

in Harvard Style

Ghavidel H., Zouaq A. and Desmarais M. (2020). Using BERT and XLNET for the Automatic Short Answer Grading Task.In Proceedings of the 12th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-417-6, pages 58-67. DOI: 10.5220/0009422400580067

in Bibtex Style

@conference{csedu20,
author={Hadi Ghavidel and Amal Zouaq and Michel Desmarais},
title={Using BERT and XLNET for the Automatic Short Answer Grading Task},
booktitle={Proceedings of the 12th International Conference on Computer Supported Education - Volume 1: CSEDU,},
year={2020},
pages={58-67},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0009422400580067},
isbn={978-989-758-417-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - Using BERT and XLNET for the Automatic Short Answer Grading Task
SN - 978-989-758-417-6
AU - Ghavidel H.
AU - Zouaq A.
AU - Desmarais M.
PY - 2020
SP - 58
EP - 67
DO - 10.5220/0009422400580067