Improving the Quiz - Student Preparation and Confidence as Feedback Metrics

Pantelis M. Papadopoulos, Antonis Natsis, Nikolaus Obwegeser

2017

Abstract

The study analyzes the potential of different feedback metrics that could improve learning in quiz-based activities. For five consecutive weeks, a group of 91 sophomore students started their classes on Information Systems with a short multiple-choice quiz. The quiz activity was organized into three phases: (a) provide initial response to the questions, (b) view feedback on class activity and revise initial responses, and (c) discuss correct answers and class performance with the teacher. The feedback included information on the percentage of students that selected each choice, on students’ self-reported levels of preparation, and their, also self-reported, confidence that their initial responses were correct. The students used an online quiz tool that was developed for the study and were randomly distributed into four groups, according to the type of feedback they received (only percentage; percentage & confidence; percentage & preparation; percentage, confidence, & preparation). Result analysis revealed that students were relying first and foremost on the percentage metric, even in cases where a wrong answer had the highest percentage value. However, statistical analysis also revealed a significant main effect for confidence and preparation metrics in questions where the percentage metric was ambiguous (i.e., several choices with high percentages).

References

  1. Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., & Koedinger, K. (2008). Why Students Engage in “Gaming the System” Behavior in Interactive Learning Environments. Journal of Interactive Learning Research. 19(2), 185-224.
  2. Bodemer, D. (2011). Tacit guidance for collaborative multimedia learning. Computers in Human Behavior, 27(3), 1079-1086.
  3. Bransford, J. D., Brown, A., & Cocking, R. (2000). How people learn: Mind, brain, experience and school. Washington, DC, National Academy Press.
  4. Buder, J. (2011). Group awareness tools for learning: Current and future directions. Computers in Human Behavior, 27, 1114-1117.
  5. Buil, I., Catalán, S., & Martínez, E. (2016). Do clickers enhance learning? A control-value theory approach. Computers & Education, 103, 170-182.
  6. Denny, P. (2013). The effect of virtual achievements on student engagement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 7813). ACM, New York, NY, USA, 763-772.
  7. Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: defining "gamification". In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. ACM, New York, 9-15.
  8. DiBattista, D., Mitterer, J. O., & Gosse, L. (2004). Acceptance by undergraduates of the immediate feedback assessment technique for multiple choice testing. Teaching in Higher Education, 9(1), 17-28.
  9. Erkens, M., Schlottbom, P., & Bodemer, D. (2016). Qualitative and Quantitative Information in Cognitive Group Awareness Tools: Impact on Collaborative Learning. In Looi, C.-K., Polman, J., Cress, U., & Reimann, P. (Eds.), Transforming Learning, Empowering Learners: 12th International Conference of the Learning Sciences (pp. 458-465). Singapore: International Society of the Learning Sciences.
  10. Janssen, J., & Bodemer, D. (2013). Coordinated computersupported collaborative learning: Awareness and awareness tools. Educational Psychologist,48, 40-55.
  11. Kleitman, S., & Costa, D. S. J. (2014). The role of a novel formative assessment tool (Stats-mIQ) and individual differences in real-life academic performance. Learning and Individual Differences, 29, 150-161.
  12. Lin, J. -W., Mai, L. -J., & Lai, Y.-C. (2015). Peer interaction and social network analysis of online communities with the support of awareness of different contexts. International Journal of ComputerSupported Collaborative Learning, 10(2), 139-159.
  13. Méndez-Coca, D., & Slisko, J. (2013). Software Socrative and smartphones as tools for implementation of basic processes of active physics learning in classroom: An initial feasibility study with prospective teachers. European Journal of Physics Education, 4(2), 17-24.
  14. Papadopoulos, P. M., Demetriadis, S. N., & Weinberger, A. (2013). “Make It Explicit!”: Improving Collaboration through Increase of Script Coercion. Journal of Computer Assisted Learning, 29 (4), 383 - 398.
  15. Papadopoulos, P. M., Lagkas, T. D., & Demetriadis, S. N. (2016). How Revealing Rankings Affects Student Attitude and Performance in a Peer Review Learning Environment. Communications in Computer and Information Science (CCIS): Computer Supported Education 2015. Vol. 583 Springer Verlag, 2016. p. 225-240.
  16. Schnaubert, L., & Bodemer, D. (2015). Subjective Validity Ratings to Support Shared Knowledge Construction in CSCL. In O. Lindwall, P. Häkkinen, T. Koschmann, P. Tchounikine, & S. Ludvigsen (Eds.), Exploring the Material Conditions of Learning: The Computer Supported Collaborative Learning (CSCL) Conference 2015 (Vol. 2) (pp. 933- 934). Gothenburg: International Society of the Learning Sciences.
  17. Sosa, G.W., Berger, D. E., Saw, A. T., &Mary, J. C. (2011). Effectiveness of computer-assisted instruction in statistics: A meta-analysis. Review of Educational Research, 81(1), 97-128.
  18. Wang, A.I. (2015). The wear out effect of a game-based student response system. Computers & Education, 82, 217-227.
  19. Wang, T.-H. (2008). Web-based quiz-game-like formative assessment: Development and evaluation. Computers & Education, 51, 1247-1263.
Download


Paper Citation


in Harvard Style

Papadopoulos P., Natsis A. and Obwegeser N. (2017). Improving the Quiz - Student Preparation and Confidence as Feedback Metrics . In Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-239-4, pages 59-69. DOI: 10.5220/0006283700590069


in Bibtex Style

@conference{csedu17,
author={Pantelis M. Papadopoulos and Antonis Natsis and Nikolaus Obwegeser},
title={Improving the Quiz - Student Preparation and Confidence as Feedback Metrics},
booktitle={Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: CSEDU,},
year={2017},
pages={59-69},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006283700590069},
isbn={978-989-758-239-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - Improving the Quiz - Student Preparation and Confidence as Feedback Metrics
SN - 978-989-758-239-4
AU - Papadopoulos P.
AU - Natsis A.
AU - Obwegeser N.
PY - 2017
SP - 59
EP - 69
DO - 10.5220/0006283700590069