Restrictive Stochastic Item Selection Methods in
Cognitive Diagnostic Computerized Adaptive Testing
Chun Wang and Hua-Hua Chang Alan Huebner
Two new item selection methods, the restrictive progressive method and restrictive threshold method were proposed in this paper.
The advantage of these methods is that they incorporate a random element in the item selection in addition to the item information, which can help equalize item exposure rates with a minimal loss of measurement accuracy.
The fusion model was used in this paper followed by an introduction of the posterior weighted Kullback-Leibler (PWKL) information index. Estimation of the fusion model has been accomplished by casting it into a hierarchical Bayesian framework and applying an Markov chain Monte Carlo algorithm.
Two simulation studies were conducted. Study 1 illustrates the utility of the two new methods by considering different values of β. In Study 2, the performance of these methods with a fixed β are compared with five other item selection methods.
The results shows the two new methods strike a better balance between item exposure control and measurement accuracy.
Some key concepts:
Item selection rules generally are based on either maximizing information about the location of the examinee in the ability space or minimizing the error in the estimation of location. There are two different information measures that are widely used in adaptive tests:
l The Fisher information:
Fisher information measures the amount of information that an observable random variable X carries about an unknown parameter θ. However, it requires that the conditional distribution of X given θ to be continuous with respect to θ; it does not lend itself directly to CD-CAT
l Kullback-Leibler (KL) information.
The KL divergence between two probability distribution is applicable to cases where the latent trait is discrete.
The formulation of KL information in CD-CAT is introduced and modified, which weighted by the posterior probability in order to reflect the varying importance of different patterns
Comments:
1 There is no H* parameter in formula 9, while in the explanation the H* appears. I think may be the author miss the letter H in that.
2 I am not quite familiar with CD-CAT, from the simulation study it seems the new methods are very good. I am prefer to the Restrictive Threshold Method.
3 What is the performance of the two methods to be used in the real data？