Comprehensive Empirical Analysis of Stop Criteria in Computerized Adaptive Testing

Patricia Gilavert, Valdinei Freire

2021

Abstract

Computerized Adaptive Testing is an assessment approach that selects questions one after another while conditioning each selection on the previous questions and answers. CAT is evaluated mainly for its precision, the correctness of estimation of the examinee trait, and efficiency, the test length. The precision-efficiency trade-off depends mostly on two CAT components: an item selection criterion and a stop criterion. While much research is dedicated to the first, stop criteria lack relevant research. We contribute with a comprehensive evaluation of stop criteria. First, we test a variety of seven stop-criteria for different setups of item banks and estimation mechanism. Second, we contribute with a precision-efficiency trade-off method to evaluate stop criteria. Finally, we contribute with an experiment considering simulations over a myriad of synthetic item banks. We conclude in favor of the Fixed-Length criterion, as long it can be tuned to the item bank at hand; the Fixed-Length criterion shows a competitive precision-efficiency trade-off curve in every scenario while presenting zero variance in test length. We also highlight that estimation mechanism and item-bank distribution have a influence over the performance of stop criteria.

Paper Citation

in Harvard Style

Gilavert P. and Freire V. (2021). Comprehensive Empirical Analysis of Stop Criteria in Computerized Adaptive Testing. In Proceedings of the 13th International Conference on Computer Supported Education - Volume 1: CSEDU, ISBN 978-989-758-502-9, pages 48-59. DOI: 10.5220/0010500200480059

in Bibtex Style

@conference{csedu21,
author={Patricia Gilavert and Valdinei Freire},
title={Comprehensive Empirical Analysis of Stop Criteria in Computerized Adaptive Testing},
booktitle={Proceedings of the 13th International Conference on Computer Supported Education - Volume 1: CSEDU,},
year={2021},
pages={48-59},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010500200480059},
isbn={978-989-758-502-9},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Computer Supported Education - Volume 1: CSEDU,
TI - Comprehensive Empirical Analysis of Stop Criteria in Computerized Adaptive Testing
SN - 978-989-758-502-9
AU - Gilavert P.
AU - Freire V.
PY - 2021
SP - 48
EP - 59
DO - 10.5220/0010500200480059