COMPARISON OF ORAL EXAMINATION AND EXAMINATION METHODS BASED ON MULTIPLE-CHOICE QUESTIONS USING PERSONAL COMPUTERS

Dimos Triantis, Charalampos Stergiopoulos, Panagiotis Tsiakas

2009

Abstract

The aim of this work was to compare the use of multiple-choice questions (MCQs) as an examination method, to the one based on oral-response questions (ORQs). The MCQs have an advantage concerning objectivity in the grading process and speed in production of results. But they also introduce an error in the final formulation of the score. The error concerns the probability of answering a question by chance or based on an instinctive feeling. In the present study, both MCQ and ORQ tests were given to examinees, in the framework of a computer-based learning system. Avoiding the procedure of mixed scoring, e.g. both positive and negative markings, a set of pairs of MCQs was composed. The MCQs in each pair were similar, produced by the same topic. This similarity was not evident for an examinee without adequate knowledge on the particular topic. The examination based on these “paired” MCQs, by using a suitable scoring rule, when made to the same sample of students, οn the same topics and with the same levels of difficulty, gave results that were statistically indistinguishable with the grades produced by an examination based on ORQs, while both the “paired” MCQ test results and the ORQ test results differed significantly from those obtained from a MCQ using positive-only scoring rule.

References

  1. Bennett, R. E., Rock, D. A,, & Wang, M. (1991). Equivalence of free-response and multiple-choice items. Journal of Educational Measurement, 28, 77- 92.
  2. Bereby-Meyer, Y., Meyer, J., Budescu, D. V. (2003).
  3. Decision making under internal uncertainty: the case of multiple-choice tests with different scoring rules, Acta Psychologica, 112, 207-220.
  4. Bridgeman, B. (1991). Essays and multiple-choice tests as predictors of college freshman GPA. Research in Higher Educafion, 32, 319-332.
  5. Castellan, N. (1993). Evaluating information technology in teaching and learning behavior. Research Methods Instruments & Computers, 25, 233-237.
  6. Crossman, D. (1997). The evolution of the World Wide Web as an emerging instructional technology tool. In Badrul H. Khan (Ed.), Web-based instruction, 19-23. N.J.: ?ducational Technology Publications.
  7. Daniel, J. S. (1996). Mega-universities and knowledge media: technology strategies for higher education. London: Keegan Press.
  8. Dede C., 2000, Emerging Technologies and Distributed Learning in Higher Education. In: D. Hanna, ed., Higher Education in an Era of Digital Competition: Choices and Challenges, New York, 2000, New York: Atwood, 71-92.
  9. Fox R., 2002, Online technologies changing university practices, In: A. Herrmann & M. M. Kulski, eds, Flexible Futures in Tertiary Teaching, 2-4 February 2000, Curtin University of Technology, Perth, WA. Perth: Curtin University of Technology, 235-41.
  10. Goggin, N.L., Finkenberg, M.E., & Morrow, J.R. (1997). Instructional technology in higher education teaching. Quest, 49(3), 280-290.
  11. Greek University Network (2000). http://www.gunet.gr.
  12. Johnston, I. (1997), The place of information technology in the teaching of physics majors, AIP Conference Proceedings, 399, 343-356.
  13. Lehmann, H., Freedman, J., Massad, J., & Dintzis, R. (1999). An ethnographic, controlled study of the use of a computer-based histology atlas during a laboratory course. Journal of American Medical Information Association, 6(1), 38-52.
  14. Open Source eLearning and eWorking platform (2001). http://www.claroline.net.
  15. Phillips, R. L. (1992). Opportunities for multimedia in education. In S.Cunningham & R. J. Hubbold (Eds), Interactive Learning through Visualization: The Impact of Computer Graphics in Education, (pp. 25- 35). Berlin: Springer- Verlag.
  16. Stergiopoulos, C., Tsiakas, P., Kaitsa, M. & Triantis, D. (2006). Evaluating Electronic Examination Methods of Students of Electronics. Effectiveness and Comparison to the Paper-and-Pencil Method, IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (SUTC 2006) (Book 2, pp. 143-149).
  17. Stergiopoulos, C., Tsiakas, P., Kaitsa, M., Triantis, D., Fragoulis, I. & Ninos, C. (2006). Methods of Electronic Examination Applied to Students of Electronics. Comparison of results with the conventional (paper-and-pencil) method, In Proceedings of the 2nd International Conference on Web Information Systems and Technologies (pp. 305- 311).
  18. Triantis, D., Stavrakas, I., Tsiakas, P., Stergiopoulos, C. & Ninos, D. (2004). A pilot Application of Electronic Examination Applied to Students of Electronic Engineering: Preliminary Results., WSEAS transactions on advances in engineering education, Vol. 1, 26-30.
  19. Wainer, H., Wang, X.-B., & Thissen, D. (1994). How well can we equate test forms that are constructed by examinees? Journal of Educational Measurement, 31, 183-199.
Download


Paper Citation


in Harvard Style

Triantis D., Stergiopoulos C. and Tsiakas P. (2009). COMPARISON OF ORAL EXAMINATION AND EXAMINATION METHODS BASED ON MULTIPLE-CHOICE QUESTIONS USING PERSONAL COMPUTERS . In Proceedings of the First International Conference on Computer Supported Education - Volume 2: CSEDU, ISBN 978-989-8111-82-1, pages 13-18. DOI: 10.5220/0001977300130018


in Bibtex Style

@conference{csedu09,
author={Dimos Triantis and Charalampos Stergiopoulos and Panagiotis Tsiakas},
title={COMPARISON OF ORAL EXAMINATION AND EXAMINATION METHODS BASED ON MULTIPLE-CHOICE QUESTIONS USING PERSONAL COMPUTERS},
booktitle={Proceedings of the First International Conference on Computer Supported Education - Volume 2: CSEDU,},
year={2009},
pages={13-18},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001977300130018},
isbn={978-989-8111-82-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the First International Conference on Computer Supported Education - Volume 2: CSEDU,
TI - COMPARISON OF ORAL EXAMINATION AND EXAMINATION METHODS BASED ON MULTIPLE-CHOICE QUESTIONS USING PERSONAL COMPUTERS
SN - 978-989-8111-82-1
AU - Triantis D.
AU - Stergiopoulos C.
AU - Tsiakas P.
PY - 2009
SP - 13
EP - 18
DO - 10.5220/0001977300130018