A Multi-dimensional Peer Assessment System

Usman Wahid, Mohamed Amine Chatti, Uzair Anwar, Ulrik Schroeder

2017

Abstract

Modern day education and learning has moved on from brick and mortar institutions to open learning environments. This recent shift in education has had its effects on the field of assessment and feedback as well. The traditional methods of assessment are being replaced by new methods from the field of open assessment. One such assessment method for open learning environments is peer assessment. Peer assessment is a crowd sourcing technique which lessens the teacher workload and gives students a voice in the assessment process. Despite being the leading assessment method in open learning environments, the tools available for peer assessment are lagging far behind. Most peer assessment tools are context and domain specific which hinders their uniform adoption across different fields of study. This paper introduces the core peer assessment module in the Peer Assessment & Learning Assessment System (PALAS). The module is a flexible, customizable and multi-dimensional peer assessment system which allows the teachers to configure the peer assessment process according to their liking and requirements without any changes required to the system design.

References

  1. Astels, D. (2003). Test Driven development: A Practical Guide. Prentice Hall Professional Technical Reference.
  2. Brooke, J. (2013). SUS: a retrospective. Journal of Usability Studies, 8(2):29-40.
  3. Chatti, M. A., Lukarov, V., Thüs, H., Muslim, A., Yousef, A. M., Wahid, U., Greven, C., Chakrabarti, A., and Schroeder, U. (2014). Learning analytics: Challenges and future research directions. eleed, 10(1).
  4. Costello, J. and Crane, D. (2013). Technologies for learnercentered feedback. Open Praxis, 5(3):217-225.
  5. Draaijer, S. and van Boxel, P. (2006). Summative peer assessment using 'turnitin' and a large cohort of students: A case study.
  6. Falchikov, N. and Goldfinch, J. (2000). Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of educational research, 70(3):287-322.
  7. Frederiksen, J. R. and Collins, A. (1989). A systems approach to educational testing. Educational researcher, 18(9):27-32.
  8. Gehringer, E. F. (2001). Electronic peer review and peer grading in computer-science courses. ACM SIGCSE Bulletin, 33(1):139-143.
  9. Gielen, S., Dochy, F., Onghena, P., Struyven, K., and Smeets, S. (2011). Goals of peer assessment and their associated quality concepts. Studies in Higher Education, 36(6):719-735.
  10. Goh, G., Lai, X., and Rajapakse, D. C. (2011). Teammates: A cloud-based peer evaluation tool for student team projects.
  11. Hamer, J., Kell, C., and Spence, F. (2007). Peer assessment using aropä. In Proceedings of the ninth Australasian conference on Computing education-Volume 66, pages 43-54. Australian Computer Society, Inc.
  12. Joordens, S., Desa, S., and Paré, D. (2009). The pedagogical anatomy of peer-assessment: Dissecting a peerscholar assignment. Journal of Systemics, Cybernetics & Informatics, 7(5).
  13. Kaufman, J. H. and Schunn, C. D. (2011). Students perceptions about peer assessment for writing: their origin and impact on revision work. Instructional Science, 39(3):387-406.
  14. Komarov, S. and Gajos, K. Z. (2014). Organic peer assessment. In Proceedings of the CHI 2014 Learning Innovation at Scale workshop.
  15. Kulkarni, C., Bernstein, M. S., and Klemmer, S. (2015). Peerstudio: Rapid peer feedback emphasizes revision and improves performance. In Proceedings from The Second (2015) ACM Conference on Learning@ Scale, pages 75-84.
  16. Kulkarni, C. E., Socher, R., Bernstein, M. S., and Klemmer, S. R. (2014). Scaling short-answer grading by combining peer assessment with algorithmic scoring. In Proceedings of the first ACM conference on Learning@ scale conference, pages 99-108. ACM.
  17. Lehmann, K. and Leimeister, J. M. (2015). Theory-driven design of an it-based peer assessment to assess high cognitive levels of educational objectives in largescale learning services. In 23rd European Conference on Information Systems (ECIS 2015), Mnster, Germany.
  18. Luo, H., Robinson, A. C., and Park, J.-Y. (2014). Peer grading in a mooc: Reliability, validity, and perceived effects. Online Learning: Official Journal of the Online Learning Consortium, 18(2).
  19. Luxton-Reilly, A. (2009). A systematic review of tools that support peer assessment. Computer Science Education, 19(4):209-232.
  20. McCrea, B. and Weil, M. (2011). On cloud nine: Cloudbased tools are giving k-12 collaboration efforts a boost. THE Journal (Technological Horizons In Education), 38(6):46.
  21. McGarr, O. and Clifford, A. M. (2013). 'just enough to make you take it seriously': exploring students' attitudes towards peer assessment. Higher education, 65(6):677-693.
  22. Osmani, A. (2012). Learning JavaScript design patterns. ” O'Reilly Media, Inc.”.
  23. O'Toole, R. (2013). Pedagogical strategies and technologies for peer assessment in massively open online courses (moocs).
  24. Planas Lladó, A., Soley, L. F., Fraguell Sansbelló, R. M., Pujolras, G. A., Planella, J. P., Roura-Pascual, N., Sun˜ol Martínez, J. J., and Moreno, L. M. (2014). Student perceptions of peer assessment: an interdisciplinary study. Assessment & Evaluation in Higher Education, 39(5):592-610.
  25. Suen, H. (2014). Peer assessment for massive open online courses (moocs). The International Review of Research in Open and Distributed Learning, 15(3).
  26. Sung, Y.-T., Chang, K.-E., Chiou, S.-K., and Hou, H.-T. (2005). The design and application of a web-based self-and peer-assessment system. Computers & Education, 45(2):187-202.
  27. Topping, K. J. (2005). Trends in peer learning. Educational psychology, 25(6):631-645.
  28. Vogelsang, T. and Ruppertz, L. (2015). On the validity of peer grading and a cloud teaching assistant system. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, pages 41-50. ACM.
  29. Vozniuk, A., Holzer, A., and Gillet, D. (2014). Peer assessment based on ratings in a social media course. In Proceedings of the Fourth International Conference on Learning Analytics And Knowledge, pages 133-137. ACM.
  30. Wahid, U., Chatti, M. A., and Schroeder, U. (2016a). Improving Peer Assessment by using Learning Analytics. In GI Edition Proceedings Band 262 DeLFI 2016 Die 14. E-Learning Fachtagung Informatik : 11.-14. September 2016 Potsdam / Ulrike Lucke, Andreas Schwill, Raphael Zender ; Gesellschaft fr Informatik (GI), Bonn, Herausgeber, pages 52-55. 14. e-Learning Fachtagung Informatik, Potsdam (Germany), 11 Sep 2016 - 14 Sep 2016, Kllen.
  31. Wahid, U., Chatti, M. A., and Schroeder, U. (2016b). A systematic analysis of peer assessment in the mooc era and future perspectives. In Proceedings of the Eighth International Conference on Mobile, Hybrid, and Online Learning, elml 2016. IARIA XPS Press.
  32. Walvoord, M. E., Hoefnagels, M. H., Gaffin, D. D., Chumchal, M. M., and Long, D. A. (2008). An analysis of calibrated peer review (cpr) in a science lecture classroom. Journal of College Science Teaching, 37(4):66.
  33. Wang, Y., Liang, Y., Liu, L., and Liu, Y. (2014). A motivation model of peer assessment in programming language learning. CoRR, abs/1401.6113.
  34. Willmot, P. and Pond, K. (2012). Multi-disciplinary peermark moderation of group work. International Journal of Higher Education, 1(1):p2.
  35. Yousef, A. M. F., Wahid, U., Chatti, M. A., Schroeder, U., and Wosnitza, M. (2015). The effect of peer assessment rubrics on learners' satisfaction and performance within a blended mooc environment. In Proc. CSEDU 2015 conference, volume 2, pages 148-159.
Download


Paper Citation


in Harvard Style

Wahid U., Chatti M., Anwar U. and Schroeder U. (2017). A Multi-dimensional Peer Assessment System . In Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: LLL, ISBN 978-989-758-239-4, pages 683-694. DOI: 10.5220/0006390006830694


in Bibtex Style

@conference{lll17,
author={Usman Wahid and Mohamed Amine Chatti and Uzair Anwar and Ulrik Schroeder},
title={A Multi-dimensional Peer Assessment System},
booktitle={Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: LLL,},
year={2017},
pages={683-694},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006390006830694},
isbn={978-989-758-239-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: LLL,
TI - A Multi-dimensional Peer Assessment System
SN - 978-989-758-239-4
AU - Wahid U.
AU - Chatti M.
AU - Anwar U.
AU - Schroeder U.
PY - 2017
SP - 683
EP - 694
DO - 10.5220/0006390006830694