Authors:
Mark Gierl
1
and
Tahereh Firoozi
2
Affiliations:
1
Measurement, Evaluation, and Data Science, Faculty of Education, University of Alberta, Edmonton, Alberta, Canada
;
2
School of Dentistry, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada
Keyword(s):
Automatic Item Generation, Testlet Generation, Item Modelling.
Abstract:
A testlet is a set of two or more items based on the same scenario. A testlet can be used to measure complex problem-solving skills that require a series or sequence of steps. A testlet is challenging to write because it requires one unique scenario and two or more items. Despite this challenge, large numbers of testlets are often required to support formative and summative computerized testing. The purpose of our study is to address the testlet item writing challenge by describing and demonstrating a systematic method that can be used to create large numbers of testlets. Our method is grounded in the three-step process associated with template-based automatic item generation. To begin, we describe a testlet-based item model. The model contains global and local variables. Global variables are unique to testlet generation because they can be used throughout the testlet, meaning that these variables can be used to place content anywhere in the testlet. Local variables, on the other han
d, are specific to each item model in the testlet and can only be used in the same item model. Next, we present four cases that demonstrate how global and local variables can be combined to generate testlets. Each case provides a practical example of how the testlet item model can be used to structure global and local variables in order to generate diverse sets of test items. We conclude by highlighting the benefits of testlet-based automatic item generation for computerized testing.
(More)