Streamlining Assessment using a Knowledge Metric

Nils Ulltveit-Moe, Sigurd Assev, Terje Gjøsæter, Halvard Øysæd

2016

Abstract

This paper proposes an efficient tool-supported methodology for marking student assignment answers according to a knowledge metric. This metric gives a coarse hint of student answer quality based on Shannon entropy. The methodology supports marking student assignments across each sub-assignment answer, and the metric sorts the answers, so that the most comprehensive textual answers typically get the highest ranking, and can be marked first. This ensures that the teacher quickly gets an overview over the range of answers, which allows for determining a consistent marking scale in order to reduce the risk of scale sliding or hitting the wrong scale level during marking. This approach is significantly faster and more consistent than using the traditional approach, marking each assignment individually.

References

  1. Barstad, V., Goodwin, M., and Gjøsaeter, T. (2014). Predicting Source Code Quality with Static Analysis and Machine Learning.
  2. Buckley, E. and Cowap, L. (2013). An evaluation of the use of Turnitin for electronic submission and marking and as a formative feedback tool from an educator's perspective. British Journal of Educational Technology, 44(4):562-570.
  3. Dretske, F. (1981). Knowledge and the Flow of Information. MIT Press.
  4. Dretske, F. (2000). Perception, Knowledge and Belief: Selected Essays. Cambridge University Press.
  5. Dretske, F. I. (1997). Naturalizing the Mind. MIT Press.
  6. Foltz, P. W., Laham, D., Landauer, T. K., Foltz, P. W., Laham, D., and Landauer, T. K. (1999). Automated Essay Scoring: Applications to Educational Technology. volume 1999, pages 939-944.
  7. Kakkonen, T., Myller, N., Timonen, J., and Sutinen, E. (2005). Automatic Essay Grading with Probabilistic Latent Semantic Analysis. In Proceedings of the Second Workshop on Building Educational Applications Using NLP, EdAppsNLP 05, pages 29-36, Stroudsburg, PA, USA. Association for Computational Linguistics.
  8. Rehder, B., Schreiner, M. E., Wolfe, M. B. W., Laham, D., Landauer, T. K., and Kintsch, W. (1998). Using latent semantic analysis to assess knowledge: Some technical considerations. Discourse Processes, 25(2- 3):337-354.
  9. Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27:379-423, 623-656.
  10. Sikora, A. S. (2015). Mathematical theory of student assessment through grading.
  11. Valenti, S., Neri, F., Cucchiarelli, A., Valenti, S., Neri, F., and Cucchiarelli, A. (2003). An Overview of Current Research on Automated Essay Grading. Journal of Information Technology Education: Research, 2(1):319-330.
  12. Zen, K., Iskandar, D., and Linang, O. (2011). Using Latent Semantic Analysis for automated grading programming assignments. In 2011 International Conference on Semantic Technology and Information Retrieval (STAIR), pages 82-88.
Download


Paper Citation


in Harvard Style

Ulltveit-Moe N., Assev S., Gjøsæter T. and Øysæd H. (2016). Streamlining Assessment using a Knowledge Metric . In Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-179-3, pages 190-197. DOI: 10.5220/0005862601900197


in Bibtex Style

@conference{csedu16,
author={Nils Ulltveit-Moe and Sigurd Assev and Terje Gjøsæter and Halvard Øysæd},
title={Streamlining Assessment using a Knowledge Metric},
booktitle={Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU,},
year={2016},
pages={190-197},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005862601900197},
isbn={978-989-758-179-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - Streamlining Assessment using a Knowledge Metric
SN - 978-989-758-179-3
AU - Ulltveit-Moe N.
AU - Assev S.
AU - Gjøsæter T.
AU - Øysæd H.
PY - 2016
SP - 190
EP - 197
DO - 10.5220/0005862601900197