Building a Formative Assessment System That is Easy to Adopt Yet Supports Long-term Improvement: A Review of the Literature and Design Recommendations

Ellis Solaiman, Joshua Barney, Martin Smith


Formative assessment has been shown to be an effective teaching tool, yet is infrequently used in practice. With the intent of building a formative e-assessment platform, we examine research on formative practices and supporting computer-based systems with a focus on: institutional barriers to adoption of previous systems; senses in which students and teachers can improve their practices across varying timescales; and collectible data (self-reported or otherwise) necessary or advantageous in supporting these processes. From this research we identify the minimal set of data which adequately supports these processes of improvement, arrive at a set of requirements and recommendations for an innovative system which collects, processes, and presents this data appropriately, and from these requirements design the architecture of an extensible electronic formative assessment system which balances the need for complex long-term analytics with that of accessibility.


  1. Albano, G. and Pepkolaj, L. (2014). Formative selfassessment to support self-driven mathematics education at university level. In International Conference on Web-Based Learning, pages 82-91. Springer.
  2. Allal, L. and Schwartz, G. (1996). Quelle place pour l'évaluation formative dans l'enseignement au cycle d'orientation? CO Infos, 178:5-8.
  3. Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18(1):5-25.
  4. Black and Wiliam (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2):139-144, 146-148.
  5. Black and Wiliam (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability (formerly: Journal of Personnel Evaluation in Education), 21(1):5-31.
  6. Black, P., Harrison, C., Lee, C., Marshall, B., and William, D. (2004). Working inside the black box: Assessment for learning in the classroom. 86(1):8-21.
  7. Blin, F. and Munro, M. (2008). Why hasn't technology disrupted academics' teaching practices? understanding resistance to change through the lens of activity theory. Computers & Education, 50(2):475-490.
  8. Cerezo, R., Sánchez-Santillán, M., Paule-Ruiz, M. P., and N ú n˜ez, J. C. (2016). Students' lms interaction patterns and their relationship with achievement: A case study in higher education. Computers & Education, 96:42- 54.
  9. Ebbinghaus, H. (1885). Über das Gedächtnis.
  10. Gehringer, E. F. (2014). A survey of methods for improving review quality. In International Conference on WebBased Learning, pages 92-97. Springer.
  11. Gikandi, J. W., Morrow, D., and Davis, N. E. (2011). Online formative assessment in higher education: A review of the literature. Computers & education, 57(4):2333- 2351.
  12. Greiff, S., Wüstenberg, S., and Avvisati, F. (2015). Computer-generated log-file analyses as a window into students' minds? a showcase study based on the pisa 2012 assessment of problem solving. Computers & Education, 91:92-105.
  13. Hall, R. and Hall, M. (2010). Scoping the pedagogic relationship between self-efficacy and web 2.0 technologies. Learning, Media and Technology, 35(3):255- 273.
  14. Hattie, J. and Timperley, H. (2007). The power of feedback. Review of educational research, 77(1):81-112.
  15. Hestenes, D., Wells, M., Swackhamer, G., et al. (1992). Force concept inventory. The physics teacher, 30(3):141-158.
  16. King, E. and Boyatt, R. (2015). Exploring factors that influence adoption of e-learning within higher education. British Journal of Educational Technology, 46(6):1272-1280.
  17. Kotter, J. P. and Cohen, D. S. (2002). The heart of change: Real-life stories of how people change their organizations. Harvard Business Press.
  18. Landauer, T. K. and Bjork, R. A. (1978). Optimum rehearsal patterns and name learning. Practical aspects of memory, 1:625-632.
  19. Lefevre, D. and Cox, B. (2016). Feedback in technologybased instruction: Learner preferences. British Journal of Educational Technology, 47(2):248-256.
  20. Leitner, S. (1974). So lernt man lernen. Herder.
  21. Lin, Y.-S., Chang, Y.-C., Liew, K.-H., and Chu, C.-P. (2015). Effects of concept map extraction and a test-based diagnostic environment on learning achievement and learners' perceptions. British Journal of Educational Technology.
  22. Looney, J. (2011). Integrating formative and summative assessment: Progress toward a seamless system? OECD Working Paper.
  23. Luaces, O., Alonso, A., Troncoso, A., Bahamonde, A., et al. (2015). Including content-based methods in peerassessment of open-response questions. In 2015 IEEE International Conference on Data Mining Workshop (ICDMW), pages 273-279. IEEE.
  24. Macfadyen, L. P. and Dawson, S. (2012). Numbers are not enough. why e-learning analytics failed to inform an institutional strategic plan. Educational Technology & Society, 15(3):149-163.
  25. MacLean, P. and Scott, B. (2011). Competencies for learning design: A review of the literature and a proposed framework. British Journal of Educational Technology, 42(4):557-572.
  26. Nicol, D. (2007). E-assessment by design: using multiplechoice tests to good effect. Journal of Further and Higher Education, 31(1):53-64.
  27. Nicol, D. J. and Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in higher education, 31(2):199-218.
  28. Rodrigues, F. and Oliveira, P. (2014). A system for formative assessment and monitoring of students' progress. Computers & Education, 76:30-41.
  29. Rogers, E. M. (1995). Diffusion of Innovations: Modifications of a Model for Telecommunications, pages 25-38. Springer Berlin Heidelberg, Berlin, Heidelberg.
  30. Romero, C. and Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6):601-618.
  31. Russell, C. (2009). A systemic framework for managing e-learning adoption in campus universities: individual strategies in context. Research in learning technology, 17(1).
  32. Seppälä, O., Malmi, L., and Korhonen, A. (2006). Observations on student misconceptions: A case study of the build-heap algorithm. Computer Science Education, 16(3):241-255.
  33. Spiro, R. et al. (1995). Cognitive flexibility, constructivism and hypertext: Random acces instruction for advanced knowledge acquisition. P. Steffe e J. Gale.
  34. Spitzer, H. F. (1939). Studies in retention. Journal of Educational Psychology, 30(9):641.
  35. Van Labeke, N., Whitelock, D., Field, D., Pulman, S., and Richardson, J. T. (2013). Openessayist: extractive summarisation and formative assessment of free-text essays.
  36. Vanderhoven, E., Raes, A., Montrieux, H., Rotsaert, T., and Schellens, T. (2015). What if pupils can assess their peers anonymously? a quasi-experimental study. Computers & Education, 81:123-132.
  37. Wiliam, D. (2006). Formative assessment: Getting the focus right. Educational Assessment, 11(3-4):283-289.
  38. Wiliam, D. and Thompson, M. (2007). Integrating assessment with learning: What will it take to make it work? Lawrence Erlbaum Associates.
  39. Wilson, M. (2009). Measuring progressions: Assessment structures underlying a learning progression. Journal of Research in Science Teaching, 46(6):716-730.
  40. Wozniak, P. A. (1990). Optimization of learning. Unpublished master's thesis, Poznan University of Technology. Poznan, Poland.
  41. Zou, X. and Zhang, X. (2013). Effect of different score reports of web-based formative test on students' selfregulated learning. Computers & Education, 66:54- 63.

Paper Citation

in Harvard Style

Solaiman E., Smith M. and Barney J. (2017). Building a Formative Assessment System That is Easy to Adopt Yet Supports Long-term Improvement: A Review of the Literature and Design Recommendations . In Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-239-4, pages 442-449. DOI: 10.5220/0006339404420449

in Bibtex Style

author={Ellis Solaiman and Martin Smith and Joshua Barney},
title={Building a Formative Assessment System That is Easy to Adopt Yet Supports Long-term Improvement: A Review of the Literature and Design Recommendations},
booktitle={Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: CSEDU,},

in EndNote Style

JO - Proceedings of the 9th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - Building a Formative Assessment System That is Easy to Adopt Yet Supports Long-term Improvement: A Review of the Literature and Design Recommendations
SN - 978-989-758-239-4
AU - Solaiman E.
AU - Smith M.
AU - Barney J.
PY - 2017
SP - 442
EP - 449
DO - 10.5220/0006339404420449