A Semi-Automatic Computer-Aided Assessment Approach for Marking and Providing Feedback Comments

Adewale Adesina, Roger G. Stone, Firat Batmaz, Ian Jones

2015

Abstract

Assessment is an essential part of the learning process. It is important for educators to provide detailed and reliable evaluations to students so that they can be better prepared for future studies and the workplace. Marking and providing formative feedback can be time-consuming and prone to errors especially when detailed analyses of students’ problem-solving steps are considered. A computer-aided marking and feedback support tool that aims at improving the efficiency and effectiveness of human marking may result in reduced marking time, improved consistency in marking, and improved feedback capabilities. This paper discusses a semi-automatic approach to marking problem-solving steps in the context of elementary school mathematics using analytical assessment rubrics. A prototype tool which implements the approach is described following recommendations based on research evidence in mathematics problem solving. The tool was evaluated in an observational study which compared marking-time efficiencies obtained using the technique with those obtained from marking done manually. The result suggests that the method has the potentials to facilitate broad feedback delivery, improve marking consistency and may save on marking time. The use of such marking and feedback support systems may contribute to the overall educational goal of more accurate and consistent assessment procedures.

References

  1. Aamodt, A., & Plaza, E. (1994). Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches. AI Communications, 7(1), 39-59.
  2. Adesina, A., Stone, R., Batmaz, F., & Jones, I. (2014). Touch Arithmetic: A process-based Computer-Aided Assessment approach for capture of problem solving steps in the context of elementary mathematics. Computers & Education, 78, 333-343.
  3. Bennett, R. E., Braswell, J., Oranje, A., Sandene, B., Kaplan, B., & Yan, F. (2008). Does it Matter if I Take My Mathematics Test on Computer? A Second Empirical Study of Mode Effects in NAEP. The Journal of Technology, Learning and Assessment, 6(9).
  4. Bescherer, Christine, Herding, Daniel, Kortenkamp, Ulrich, Muller, Wolfgang, & Zimmermann, Marc. (2011). E-Learning Tools with Intelligent Assessment and Feedback for Mathematics Study. Idea Group Inc (IGI).
  5. Black, P., & Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74.
  6. Brown, G., Irving, E., & Keegan, P. (2008). An Introduction to Educational Assessment, Measurement and Evaluation. Pearson Education New Zealand Limited.
  7. Bull, J., & McKenna, C. (2004). Blueprint for computerassisted assessment. London: RoutledgeFalmer.
  8. Carpenter, T. P., & Moser, J. M. (1984). The Acquisition of Addition and Subtraction Concepts in Grades One through Three. Journal for Research in Mathematics Education, 15(3), 179-202.
  9. Charles, R., Lester, Frank, & O'Daffer, Phares. (1987). How To Evaluate Progress in Problem Solving. National Council of Teachers of Mathematics, 1906 Association Drive, Reston, VA 22901.
  10. Conole, G., & Warburton, B. (2005). A review of computer assisted assessment. ALT-J, 13(1), 17-31.
  11. Dinham, S., 2008. How to Get Your School Moving and Improving: An Evidence-based Approach. Aust Council for Ed Research.
  12. Garfield, J. . B. (1994). Beyond testing and grading: Using assessment to improve student learning. Journal of Statistics Education, 2(1), 1-11.
  13. Gray, E.M., Tall, D.O., 1994. Duality, Ambiguity, and Flexibility: A “Proceptual” View of Simple Arithmetic. J. Res. Math. Educ. 25, 116-140.
  14. Hattie, J. (2008). Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. Routledge.
  15. Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112.
  16. Herding, D., & Schroeder, U. (2011). Using Capture & Replay for Semi-automatic Assessment. In CAA 2011 International Conference.
  17. Hollingsworth, J. (1960). Automatic graders for programming classes. Commun. ACM, 3(10), 528- 529.
  18. JISC. (2010). Effective Assessment in a Digital Age: A guide to technology-enhanced assessment and feedback. Retrieved 2 February 2014, from http:// www.jisc.ac.uk/media/documents/programmes/elearni ng/digiassass_eada.pdf.
  19. Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130-144.
  20. Jordan, S. (2013). E-assessment: Past, present and future. New Directions, 9(1), 87-106.
  21. Mory, E. H. (2004). Feedback Research Revisited. In Handbook of Research on Educational Communications and Technology (2nd ed.) (pp. 745- 783). Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers.
  22. National Council of Teachers in Mathematics (NCTM). (2000). Principles and Standards for School Mathematics. Reston, VA: NCTM.
  23. Orrell, J. (2008). Assessment beyond belief: the cognitive process of grading. Balancing Dilemmas in Assessment and Learning in Contemporary Education, 251-263.
  24. Richter, M. M., & Rosina, O. (2013). Case-based reasoning: a textbook. London: Springer.
  25. Rittle-Johnson, B., & Alibali, M. W. (1999). Conceptual and procedural knowledge of mathematics: Does one lead to the other? Journal of Educational Psychology, 91(1), 175-189.
  26. Sangwin, C. (2013). Computer Aided Assessment of Mathematics. Oxford University Press.
  27. Sargent, J, Wood, M.M, & Aderson, S.M. (2004). A human-computer collaborative approach to the marking of free text answers. In 8th International Computer Assisted Assessment Conference,. Loughborough University.
  28. Stevens, D. D., & Levi, A. J. (2011). Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and Promote Student Learning. Stylus Publishing, LLC.
  29. Suskie, L. (2010). Assessing Student Learning: A Common Sense Guide. John Wiley & Sons.
  30. Szetela, W. (1992). Evaluating Problem Solving in Mathematics. Educational Leadership, 49(8), 42-45.
Download


Paper Citation


in Harvard Style

Adesina A., Stone R., Batmaz F. and Jones I. (2015). A Semi-Automatic Computer-Aided Assessment Approach for Marking and Providing Feedback Comments . In Proceedings of the 7th International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-758-108-3, pages 93-100. DOI: 10.5220/0005447000930100


in Bibtex Style

@conference{csedu15,
author={Adewale Adesina and Roger G. Stone and Firat Batmaz and Ian Jones},
title={A Semi-Automatic Computer-Aided Assessment Approach for Marking and Providing Feedback Comments},
booktitle={Proceedings of the 7th International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2015},
pages={93-100},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005447000930100},
isbn={978-989-758-108-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - A Semi-Automatic Computer-Aided Assessment Approach for Marking and Providing Feedback Comments
SN - 978-989-758-108-3
AU - Adesina A.
AU - Stone R.
AU - Batmaz F.
AU - Jones I.
PY - 2015
SP - 93
EP - 100
DO - 10.5220/0005447000930100