Comparing Electronic Examination Methods for Assessing Engineering Students - The Case of Multiple-Choice Questions and Constructed Response Questions

Dimos Triantis, Errikos Ventouras, Ioanna Leraki, Charalampos Stergiopoulos, Ilias Stavrakas, George Hloupis

2014

Abstract

The aim of this work is the comparison of two well-known examination methods, the first consisted of multiple-choice questions (MCQs) and the second based on constructed-response questions (CRQs). During this research MCQ and CRQ tests were created for examining the undergraduate engineering module of “project management” and were given to a group of students. Computers and a special software package were used to support the process. During the first part the examinees had to answer a set of CRQs. Afterwards, they had to answer a set of MCQs. Both sets covered the same topics and had the same level of difficulty. The second method (MCQs) is more objective in terms of grading, though it may conceal an error in the final formulation of the score when a student gives an answer based on an instinctive feeling. To eliminate this problem a set of MCQs pairs was composed taking care that each question of the pair addressed the same topic in a way that the similarity would not be evident to a student who did not possess adequate knowledge. By applying a suitable scoring rule to the MCQs, very similar results are obtained when comparing these two examination methods.

References

  1. Bereby-Meyer, Y., Meyer, J., & Flascher, O. M. 2002. Prospect theory analysis of guessing in multiple choice tests. Journal of Behavioral Decision Making, 15(4), pp. 313-327.
  2. Bereby-Meyer, Y., Meyer, J., & Budescu, D. V. 2003. Decision making under internal uncertainty: The case of multiple-choice tests with different scoring rules. Acta psychologica, 11(2), pp. 207-220.
  3. Bush, M. E. 2006. Quality assurance of multiple-choice tests. Quality Assurance in Education, 14(4), pp. 398- 404.
  4. DeBord, K. A., Aruguete, M. S., & Muhlig, J. 2004. Are computer-assisted teaching methods effective?, Teaching of Psychology, 31(1), pp. 65-68.
  5. Dede, C. 2005. Planning for neomillennial learning styles. Educause Quarterly, 28(1), 7-12.
  6. Friedl, R., Höppler, H., Ecard, K., Scholz, W., Hannekum, A., Öchsner, W., & Stracke, S. 2006. Multimediadriven teaching significantly improves students' performance when compared with a print medium, The Annals of thoracic surgery, 81(5), pp. 1760-1766.
  7. Freeman, R., & Lewis, R. 1998. Planning and implementing assessment. Routledge.
  8. Lukhele, R., Thissen, D., & Wainer, H. 1994. On the Relative Value of Multiple Choice, Constructed Response, and Examinee Selected Items on Two Achievement Tests, Journal of Educational Measurement, 31(3), pp. 234-250.
  9. Reiser, R. A., & Dempsey, J. V. 2011. Trends and issues in instructional design and technology. Pearson.
  10. Scharf, E. M., & Baldwin, L. P. 2007. Assessing multiple choice question (MCQ) tests-a mathematical perspective, Active Learning in Higher Education, 8(1), pp. 31-47.
  11. Stergiopoulos, C., Tsiakas, P., Triantis, D., & Kaitsa, M. 2006. Evaluating Electronic Examination Methods Applied to Students of Electronics. Effectiveness and Comparison to the Paper-and-Pencil Method. In Sensor Networks, Ubiquitous, and Trustworthy Computing, 2006. IEEE International Conference on , 2, pp. 143-151..
  12. Tsiakas, P., Stergiopoulos, C., Nafpaktitis, D., Triantis, D., & Stavrakas, I. 2007. Computer as a tool in teaching, examining and assessing electronic engineering students. In EUROCON'07, The International Conference on "Computer as a Tool", pp. 2490-2497.
  13. Triantis, D., & Ventouras, E. 2011. Enhancing Electronic Examinations through Advanced Multiple-Choice Questionnaires. Higher Education Institutions and Learning Management Systems: Adoption and Standardization, 178.
  14. Ventouras, E., Triantis, D., Tsiakas, P., & Stergiopoulos, C. 2010. Comparison of examination methods based on multiple-choice questions and constructed-response questions using personal computers, Computers & Education, 54(2), pp. 455-461.
  15. Ventouras, E., Triantis, D., Tsiakas, P., & Stergiopoulos, C. 2011. Comparison of oral examination and electronic examination using paired multiple-choice questions. Computers & Education, 56(3), pp. 616- 624.
  16. Wainer, H., & Thissen, D. 1993. Combining multiplechoice and constructed-response test scores: Toward a Marxist theory of test construction. Applied Measurement in Education, 6(2), pp. 103-118.
  17. Wainer, H., & Thissen, D. 2001. True score theory: The traditional method. Test scoring, 23-72.
Download


Paper Citation


in Harvard Style

Triantis D., Ventouras E., Leraki I., Stergiopoulos C., Stavrakas I. and Hloupis G. (2014). Comparing Electronic Examination Methods for Assessing Engineering Students - The Case of Multiple-Choice Questions and Constructed Response Questions . In Proceedings of the 6th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-020-8, pages 126-132. DOI: 10.5220/0004848401260132


in Bibtex Style

@conference{csedu14,
author={Dimos Triantis and Errikos Ventouras and Ioanna Leraki and Charalampos Stergiopoulos and Ilias Stavrakas and George Hloupis},
title={Comparing Electronic Examination Methods for Assessing Engineering Students - The Case of Multiple-Choice Questions and Constructed Response Questions},
booktitle={Proceedings of the 6th International Conference on Computer Supported Education - Volume 1: CSEDU,},
year={2014},
pages={126-132},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004848401260132},
isbn={978-989-758-020-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - Comparing Electronic Examination Methods for Assessing Engineering Students - The Case of Multiple-Choice Questions and Constructed Response Questions
SN - 978-989-758-020-8
AU - Triantis D.
AU - Ventouras E.
AU - Leraki I.
AU - Stergiopoulos C.
AU - Stavrakas I.
AU - Hloupis G.
PY - 2014
SP - 126
EP - 132
DO - 10.5220/0004848401260132