A Fuzzy System to Automatically Evaluate and Improve Fairness of Multiple-Choice Questions (MCQs) based Exams

Ibrahim A. Hameed

2016

Abstract

Examination is one of the common assessment methods to assess the level of knowledge of students. Assessment methods probably have a greater influence on how and what students learn than any other factor. Assessment is used to discriminate not only between different students but also between different levels of thinking. Due to the increasing trends in class sizes and limited resources for teaching, the need arises for exploring other assessment methods. Multiple-Choice Questions (MCQs) have been highlighted as the main way of coping with the large group teaching, ease of use, testing large number of students on a wide range of course material, in a short time and with low grading costs. MCQs have been criticised for encouraging surface learning and its unfairness. MCQs have a variety of scoring options; the most widely used method is to compute the score by only focusing on the responses that the student made. In this case, the number of correct responses is counted, the number of incorrect answers is counted and a final score is reported as either the number of the correct answers or the number of correct answers minus the number of incorrect answers. The disadvantages of this approach are that other dimensions such as importance and complexity of questions are not considered, and in addition, it cannot discriminate between students with equal total score. In this paper, a method to automatically evaluate MCQs considering importance and complexity of each question and providing a fairer way to discriminating between students with equal total scores is presented.

References

  1. Angelo, T.A., Cross, K.P., 1993. Classroom assessment techniques: A handbook for college teachers. San Francisco: Jossey-Bass, 2nd edition.
  2. Boud, D., 1988. Developing student autonomy in learning (2nd ed). London: Kogan Page.
  3. Brady, A.M., 2005. Assessment of learning with multiplechoice questions. Nurse Education in Practice, 5, 238- 242.
  4. Brown, S., 1992. Trends in assessment, in: R. Harden, I. Hart & H. Mulholland (Eds) Approaches to the Assessment of Clinical Competence, Vol. 1 (Dundee, Centre for Medical Education), pp. 3-8.
  5. Donnelly, C., 2014. The use of case based multiple choice questions for assessing large group teaching: implications on student's learning. Irish Journal of Academic Practice, 3(1), Art. 12, 1-15.
  6. Francis, E.M., 2014. Teaching higher order thinking and depth of knowledge. Retrieved in Feb. 7th 2016 from http://maverikeducation.blogspot.com/2014/03/difficul ty-vs-complexity-whats.html.
  7. Fuzzy Logic Toolbox™ 2.2.7, 2016, retrieved from http://www.mathworks.com/products/fuzzylogic/ in February 6th 2016 (license no. 914603).
  8. Hameed, I.A., 2011. Using Gaussian membership functions for improving the reliability and robustness of students' evaluation systems. Expert systems with Applications, 38(6), 7135-7142.
  9. Hameed, I.A., Sørensen, C.G., 2010. Fuzzy Systems in Education: A More Reliable System for Student Evaluation, Fuzzy Systems, Azar A. T. (Ed.), ISBN: 978-953-7619-92-3, INTECH, Croatia.
  10. Mamdani, E.H., Assilian, S., 1975. An experiment in linguistic synthesis with a fuzzy logic controller. International Journal of Man-Machine Studies, 7(1), 1-13.
  11. McCoubrie, P., 2004. Improving the fairness of multiplechoice questions: a literature review. Medical Teacher, Vol. 26, No. 8, 2004, pp. 709-712.
  12. Omari, A., 2013. An Evaluation and Assessment System for Online MCQ's Exams. International Journal of Electronics and Electrical Engineering, 1(3), 219-222, doi: 10.12720/ijeee.1.3.219-222.
  13. Pamplett, R., Farnill, D., 1995. Effect of anxiety on performance in multiple choice examination. Medical Education, 29, 298-302.
  14. Saleh, I., Kim, S.-I., 2009. A fuzzy system for evaluating students' learning achievement. Expert systems with Applications, 36(3), 6236-6243.
  15. Scarino, A., 2013. Language assessment literacy as selfawareness: Understanding the role of interpretation in assessment and in teacher learning. Language Testing, 30(3), 309-327.
  16. Wiliam, D., 2011. Bryggan mellan undervisning och lärande (The bridge between teaching and learning). Lärarförbundets Förlag, Pedagogiska Magasinet, 113- 120 (its English version is retrieved from http://www.dylanwiliam.org/Dylan_Wiliams_website/ Papers_files/Pedagogiska%20magasinet%20article.do cx in 29 December 2015).
  17. Zadeh, L.A., 1973. Outline of a new approach to the analysis of complex systems and decision processes. IEEE Transactions on Systems, Man, and Cybernetics, 3(1), 28-44.
Download


Paper Citation


in Harvard Style

Hameed I. (2016). A Fuzzy System to Automatically Evaluate and Improve Fairness of Multiple-Choice Questions (MCQs) based Exams . In Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-179-3, pages 476-481. DOI: 10.5220/0005897204760481


in Bibtex Style

@conference{csedu16,
author={Ibrahim A. Hameed},
title={A Fuzzy System to Automatically Evaluate and Improve Fairness of Multiple-Choice Questions (MCQs) based Exams},
booktitle={Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU,},
year={2016},
pages={476-481},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005897204760481},
isbn={978-989-758-179-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - A Fuzzy System to Automatically Evaluate and Improve Fairness of Multiple-Choice Questions (MCQs) based Exams
SN - 978-989-758-179-3
AU - Hameed I.
PY - 2016
SP - 476
EP - 481
DO - 10.5220/0005897204760481