Optimal Test Design With Rule-Based Item Generation
Hanneke Geerlings, Wim J. van der Linden and Cees A. W. Glas
This article explores the integration of rule-base item generation and automated test assembly. The hierarchical item response theory (IRT) modeling was used which is a regular response model together with a different second-level model for its distribution of item parameters. Three different cases of item generation is discussed: (a) test assembly from a pool of pregenerated, individually calibrated items; (b) test generation on the fly from pools with calibrated item families; and (c) test generation on the fly using calibrated radicals that define the item families
The results show the variability of the item parameters within a family has a potentially large impact on the value of the family-information function. With increasing variance of the item parameters per family, the information in a test produced by the former increased.
There are some terms which are not easy for me to understand. For example, radicals and incidentals are a bit of abstract. What item feature can be defined as radicals, which are incidentals? In my own understanding the radicals may be the arithmetic operation for the items which are the test want to measure, while the incidental are different types items based on the arithmetic operation. While in some real situation like writing, we may not use the methods due to the interaction between radicals and incidentals if my understanding is right. And some assumptions are needed while real situation we can NOT meet.
The use of the method is still desirable, if it can truly separately the effect of radicals and the incidentals which may be useful in the psychological field.