AUTOMATIC ASSESSMENT OF SHORT FREE TEXT ANSWERS

Fátima Rodrigues, Lília Araújo

2012

Abstract

Assessment plays a central role in any educational process, because it is a common way to evaluate the students’ knowledge regarding the concepts related to learning objectives. Computer assisted assessment is a research branch established to study how computers can be used to automatically evaluate students’ answers. Computer assisted assessment systems developed so far, are based on a multitude of different techniques, such as Latent Semantic Analysis, Natural Language Processing and Artificial Intelligence, among others. These approaches require a reasonable corpus to start with, and depending on the domain, the corpus may require regular updates. In this paper we address the assessment of short free text answers by developing a system that captures the way the teacher evaluates the answer. For that, the system first classifies the teacher question by type. Then concerning the type of question, the system permits the teacher define scores associated with subparts of the answer. Finally, the system performs the assessment based on these sub scores. For certain types of questions, paraphrases of answers are also considered in an attempt to obtain a more precise assessment. The system was trained and tested on exams manually graded by a History teacher. Based on the results obtained, we show that there is a good correlation between the evaluation of the instructor and the evaluation performed by our system.

References

  1. Mason, O., Grove-Stephenson, I., 2002. Automated free text marking with paperless school. In Proceedings of the Sixth International Computer Assisted Assessment Conference, Loughborough University, UK.
  2. Thomas P., Haley D., Roeck A., Petre M., 2004. EAssessment using Latent Semantic Analysis in the Computer Science Domain: A Pilot Study. In Proceedings of the Workshop on eLearning for Computational Linguistics and Computational Linguistics for eLearning, pp. 38-44. Association for Computational Linguistics.
  3. Valenti S., Neri F., Cucchiarelli A., 2003. An Overview of Current Research on Automated Essay Grading. Journal of Information Technology Education.
  4. Perez-Marin D., Pascual-Nieto I., Rodriguez P., 2009. Computer-assisted assessment of free-text answers. The Knowledge Engineering Review, 24(4), pp. 353- 374.
  5. Page, E. B., 1994. New computer grading of student prose, using modern concepts and software. Journal of Experimental Education, 62(2), pp. 127-142.
  6. Jerrams-Smith J., Soh V., Callear D., 2001. Bridging gaps in computerized assessment of texts. In Proceedings of the International Conference on Advanced Learning Technologies, pp.139-140.
  7. Whittington, D., Hunt, H., 1999. Approaches to the computerized assessment of free text responses. In Proceedings of the Sixth International Computer Assisted Assessment Conference, Loughborough University, UK.
  8. Burstein J., Leacock C., Swartz R., 2001. Automated evaluation of essay and short answers. In Proceedings of the Sixth International Computer Assisted Assessment Conference, Loughborough University, UK.
  9. Rudner L. M., Liang T., 2002. Automated essay scoring using Bayes' Theorem. The Journal of Technology, Learning and Assessment, 1(2), pp. 3-21.
  10. Mitchell T., Russel T., Broomhead P., Aldridge N., 2002. Towards robust computerized marking of free-text responses. In Proceedings of the Sixth International Computer Assisted Assessment Conference, Loughboroug University, UK.
  11. Smith, J., 1998. The book, The publishing company. London, 2nd edition.
  12. Almeida, J. J., Simões A., 2007. Jspellando nas morfolimpíadas: Sobre a participação do Jspell nas morfolimpíadas. In Avaliação conjunta: um novo paradigma no processamento computacional da língua portuguesa. IST Press.
  13. OpenThesaurus, 2010 http://openthesaurus.caixamagica. pt/, last access, Feb 2011
  14. Salton G., Wong A., Yang C. S., 1975. A Vector Space Model for Automatic Indexing. Communications of the ACM, vol. 18, nr. 11, pp. 613-620.
  15. Noorbehbahani F., Kardan A. A. 2011. The Automatic assessement of free text answers using a modified BLEU algorithm. Computers & Education 56, pp.337- 345.
Download


Paper Citation


in Harvard Style

Rodrigues F. and Araújo L. (2012). AUTOMATIC ASSESSMENT OF SHORT FREE TEXT ANSWERS . In Proceedings of the 4th International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-8565-07-5, pages 50-57. DOI: 10.5220/0003920800500057


in Bibtex Style

@conference{csedu12,
author={Fátima Rodrigues and Lília Araújo},
title={AUTOMATIC ASSESSMENT OF SHORT FREE TEXT ANSWERS},
booktitle={Proceedings of the 4th International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2012},
pages={50-57},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003920800500057},
isbn={978-989-8565-07-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 4th International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - AUTOMATIC ASSESSMENT OF SHORT FREE TEXT ANSWERS
SN - 978-989-8565-07-5
AU - Rodrigues F.
AU - Araújo L.
PY - 2012
SP - 50
EP - 57
DO - 10.5220/0003920800500057