Pair-Wise: Automatic Essay Evaluation using Word Mover’s Distance

Tsegaye Misikir Tashu, Tomáš Horváth

2018

Abstract

Automated essay evaluation (AEE) represents not only as a tool to assess evaluate and score essays, but also helps to save time, effort and money without lowering the quality of goals and objectives of educational assessment. Even if the field has been developing since the 1960s and various algorithms and approaches have been proposed to implement AEE systems, most of the existing solutions give much more focus on syntax, vocabulary and shallow content measurements and only vaguely understand the semantics and context of the essay. To address the issue with semantics and context, we propose pair-wise semantic similarity essay evaluation by using the Word Mover’s Distance. This method relies on Neural Word Embedding to measure the similarity between words. To be able to measure the performance of AEE, a qualitative accuracy measure based on pairwise ranking is proposed in this paper. The experimental results show that the AEE approach using Word Mover’s distance achieve higher level of accuracy as compared to others baselines.

Paper Citation

in Harvard Style

Misikir Tashu T. and Horváth T. (2018). Pair-Wise: Automatic Essay Evaluation using Word Mover’s Distance.In Proceedings of the 10th International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-758-291-2, pages 59-66. DOI: 10.5220/0006679200590066

in Bibtex Style

@conference{csedu18,
author={Tsegaye Misikir Tashu and Tomáš Horváth},
title={Pair-Wise: Automatic Essay Evaluation using Word Mover’s Distance},
booktitle={Proceedings of the 10th International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2018},
pages={59-66},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006679200590066},
isbn={978-989-758-291-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 10th International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - Pair-Wise: Automatic Essay Evaluation using Word Mover’s Distance
SN - 978-989-758-291-2
AU - Misikir Tashu T.
AU - Horváth T.
PY - 2018
SP - 59
EP - 66
DO - 10.5220/0006679200590066