The Impact of Ranking Information on Students’ Behavior and Performance in Peer Review Settings

Pantelis M. Papadopoulos, Thomas D. Lagkas, Stavros N. Demetriadis

2015

Abstract

The paper explores the potential of usage and ranking information in increasing student engagement in a double-blinded peer review setting, where students are allowed to select freely which/how many peer works to review. The study employed 56 volunteering sophomore students majoring in Informatics and Telecommunications Engineering. We performed a controlled experiment, grouping students into 3 study conditions: control, usage data, usage and ranking data. Students in the control condition did not receive additional information. Students in the next two conditions were able to see their usage data (logins, peer work viewed/reviewed, etc.), while students in the last group could additionally see their ranking in their group according to their usage data. Results showed that while the three groups were comparable, a range of different attitudes were visible in the Ranking group. Students with more positive attitude towards ranking were more engaged and outperformed their fellow students in their group.

References

  1. Anderson, L. W., & Krathwohl, D. R. (Eds.) (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. NY: Longman.
  2. Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., & Koedinger, K. (2008). Why Students Engage in “Gaming the System” Behavior in Interactive Learning Environments. Journal of Interactive Learning Research. 19 (2), pp. 185-224. Chesapeake, VA: AACE.
  3. Demetriadis, S. N., Papadopoulos, P. M., Stamelos, I. G., & Fischer, F. (2008). The Effect of Scaffolding Students' Context-Generating Cognitive Activity in Technology-Enhanced Case-Based Learning. Computers & Education, 51 (2), 939-954. Elsevier.
  4. Denny, P. (2013). The effect of virtual achievements on student engagement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 2013.
  5. Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: defining "gamification". In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (MindTrek 7811). ACM, New York, NY, USA, 9-15.
  6. Goldin, I. M., & Ashley, K. D., (2011). Peering Inside Peer-Review with Bayesian Models. In G. Biswas et al. (Eds.): AIED 2011, LNAI 6738, pp. 90-97. Springer-Verlag: Berlin.
  7. Hansen, J., & Liu, J. (2005). Guiding principles for effective peer response. ELT Journal, 59, 31-38.
  8. Li, L., Liu, X. & Steckelberg, A.L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525-536.
  9. Liou, H. C., & Peng, Z. Y. (2009). Training effects on computer-mediated peer review. System, 37, 514-525.
  10. Lundstrom, K., & Baker, W. (2009). To give is better than to receive: The benefits of peer review to the reviewer's own writing. Journal of Second Language Writing, 18, 30-43.
  11. Luxton-Reilly, A. (2009). A systematic review of tools that support peer assessment. Computer Science Education, 19:4, 209-232.
  12. Martinez-Mones, A., Gomez-Sanchez, E., Dimitriadis, Y.A., Jorrin-Abellan, I.M., Rubia-Avi, B., & VegaGorgojo, G. (2005). Multiple Case Studies to Enhance Project-Based Learning in a Computer Architecture Course. Education, IEEE Transactions on, 48(3), 482- 489.
  13. McConnell, J. (2001). Active and cooperative learning. Analysis of Algorithms: An Active Learning Approach. Jones & Bartlett Pub.
  14. Norris, M. & Pretty, S. (2000). Designing the Total Area Network. New York: Wiley.
  15. Papadopoulos, P. M., Demetriadis, S. N., & Stamelos, I. G. (2009). Analyzing the Role of Students' SelfOrganization in Scripted Collaboration: A Case Study. Proceedings of the 8th International Conference on Computer Supported Collaborative Learning - CSCL 2009 (pp. 487-496), Rhodes, Greece. ISLS.
  16. Papadopoulos, P. M., Lagkas, T. D., & Demetriadis, S. N., (2012). How to Improve the Peer Review Method: Free-Selection vs Assigned-Pair Protocol Evaluated in a Computer Networking Course. Computers & Education, 59, 182 - 195. Elsevier.
  17. Papadopoulos, P. M., Lagkas, T. D., & Demetriadis, S. N., (2015). Intrinsic versus Extrinsic Feedback in PeerReview: The Benefits of Providing Reviews (under review).
  18. Scardamalia, M., & Bereiter, C. (1994). Computer support for knowledge-building communities. The Journal of the Learning Sciences, 3(3), 265-283.
  19. Turner, S., Pérez-Quiñones, M. A., Edwards, S., & Chase, J. (2010). Peer review in CS2: conceptual learning. In Proceedings of SIGCSE'10, March 10-13, 2010, Milwaukee, Wisconsin, USA.
Download


Paper Citation


in Harvard Style

M. Papadopoulos P., D. Lagkas T. and N. Demetriadis S. (2015). The Impact of Ranking Information on Students’ Behavior and Performance in Peer Review Settings . In Proceedings of the 7th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-107-6, pages 139-147. DOI: 10.5220/0005472801390147


in Bibtex Style

@conference{csedu15,
author={Pantelis M. Papadopoulos and Thomas D. Lagkas and Stavros N. Demetriadis},
title={The Impact of Ranking Information on Students’ Behavior and Performance in Peer Review Settings},
booktitle={Proceedings of the 7th International Conference on Computer Supported Education - Volume 1: CSEDU,},
year={2015},
pages={139-147},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005472801390147},
isbn={978-989-758-107-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - The Impact of Ranking Information on Students’ Behavior and Performance in Peer Review Settings
SN - 978-989-758-107-6
AU - M. Papadopoulos P.
AU - D. Lagkas T.
AU - N. Demetriadis S.
PY - 2015
SP - 139
EP - 147
DO - 10.5220/0005472801390147