KASS: Korean Automatic Scoring System for Short-answer Questions

Eun-Seo Jang, Seung-Shik Kang, Eun-Hee Noh, Myung-Hwa Kim, Kyung-Hee Sung, Tae-Je Seong

2014

Abstract

The scoring of short-answer questions in a national-wide achievement test to public school students needs a lot of human efforts and financial expenses. Since we know that natural language processing technology can be applied to replace the manual scoring process by automatic scoring software, many researchers tried to build an automatic scoring system like c-rater and e-rater in English. In this paper, we explored a Korean automatic scoring system for short and free-text responses. NLP techniques like morphological analysis are used to build a token-based scoring template for increasing the coverage of the automatic scoring process. We performed an experiment to measure the efficiency of the automatic scoring system and it covered about 90 to 95% of the student responses with an agreement rate 95% to the manual scoring.

References

  1. E. B. Page, 1966. The imminence of grading essays by computer, Phi Delta Kappa, pp.238-243.
  2. S. Valenti, F. Neri, and A. Cucchiarelli, 2003. An overview of current research on automated essay grading, Journal of Information Technology Education, pp.319-330.
  3. S. Dikli, 2006. An overview of automated scoring of essays, The journal of technology, learning, and assessment, pp.1-35.
  4. S. Elliot, 2003. Intellimetric: From here to validity, Automated essay scoring: A cross-disciplinary perspective, pp.71-86.
  5. M. D. Shermis, J. C. Burstein, 2003. Automated essay scoring: A cross-disciplinary perspective, Lawrence Erlbaum Associates, Inc.
  6. C. Leacock and M. Chodorow, 2003. C-rater: automated scoring of short-answer questions, Computers and the Humanities, vol.37, pp.389-405.
  7. J. Sukkarieh and J. Blackmore, 2009. C-rater: automatic content scoring for short constructed response, Proceedings of the Twenty-Second International FLAIRS Conference, pp.290-295.
  8. T. J. Sung, K. S. Yang, T. H. Kang, and E. Y. Chung, 2010. A survey of computer-based scoring method of free-text or constructed answers in study-advancement evaluation, Korea Institute for Curriculum and Evaluation, Research Report RRE 2010-1.(in Korean).
  9. E. H. Noh, J. H. Sim, M. H. Kim, and J. H. Kim, 2012. A prospect of the developing an automatic scoring system for Korean free-text answers in a large-scale evaluation, Korea Institute for Curriculum and Evaluation, Research Report ORM 2012-92.(in Korean).
  10. E. M. Chung, M. S. Choi, and J. C. Shim, 2009. Design and implementation of automatic marking system for a subjectivity problem of the program, Journal of Korea Multimedia Society, vol.12, no.5, pp.767-776.(in Korean).
  11. H. J. Park and W. S. Kang, 2003. Design and implementation of a subjective-type evaluation system using natural language processing technique, The Journal of Korean association of computer education, vol.6, no.3, pp.207-216.(in Korean).
  12. W. S. Kang, 2011. Automatic grading system for subjective questions through analyzing question type, The Journal of the Korea Contents Association, vol.11, no.2, pp.13-21. (in Korean).
  13. J. S. Oh, W. J. Cho, Y. S. Kim, and J. Y. Lee, 2005. A descriptive question marking system based on semantic kernels, The Journal of Korean institute of information technology, vol.3, no.4, pp.95-104. (in Korean).
  14. W. J. Cho, 2006. An Intelligent Marking System based on Semantic Kernel and Korean WordNet, Master Thesis, Hallym University.(in Korean).
  15. I. N. Park, E. H. Noh, J. H. Sim, M. H. Kim, and S. S. Kang, 2012. Answer template description for automatic scoring of Korean free-text or constructed answers, Proceedings of the 24th Hangul and Korean Language Processing, pp.138-141.(in Korean).
  16. I. N. Park, E. H. Noh, J. H. Sim, M. H. Kim, and S. S. Kang, 2012. Concept-based automatic scoring system for Korean free-text or constructed answers, Proceedings of the 24th Hangul and Korean Language Processing, pp.69-72. (in Korean).
  17. S. S. Kang and Y. T. Kim, 1994. Syllable-based model for the Korean morphology, Proceedings of the 15th International Conference on Computational Linguistics (COLING-94), vol.1, Kyoto, Japan, pp.221-226.
Download


Paper Citation


in Harvard Style

Jang E., Kang S., Noh E., Kim M., Sung K. and Seong T. (2014). KASS: Korean Automatic Scoring System for Short-answer Questions . In Proceedings of the 6th International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-758-021-5, pages 226-230. DOI: 10.5220/0004864302260230


in Bibtex Style

@conference{csedu14,
author={Eun-Seo Jang and Seung-Shik Kang and Eun-Hee Noh and Myung-Hwa Kim and Kyung-Hee Sung and Tae-Je Seong},
title={KASS: Korean Automatic Scoring System for Short-answer Questions},
booktitle={Proceedings of the 6th International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2014},
pages={226-230},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004864302260230},
isbn={978-989-758-021-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - KASS: Korean Automatic Scoring System for Short-answer Questions
SN - 978-989-758-021-5
AU - Jang E.
AU - Kang S.
AU - Noh E.
AU - Kim M.
AU - Sung K.
AU - Seong T.
PY - 2014
SP - 226
EP - 230
DO - 10.5220/0004864302260230