Restrictive Stochastic Item Selection Methods in Cognitive Diagnostic Computerized Adaptive Testing
This paper developed two new item selection methods which incorporated progressive and proportional ideas into the posterior weight Kullback-Leibler information method. These two methods used the beta parameter to produce different weights to adjust the importance between randomized and information, in which the idea is similar to Barrada et al.’s (2008) progressive method and proportional method. The authors manipulated these two item selection methods with different beta parameters and test lengths under CD-CAT algorithm. It found that these two methods could be applied quite well under CD-CAT from the first simulation. That is, recovery of all attributes and individual attribute well. For the second simulation, authors adopted these two new methods along with the other five methods (PWKL, Barrada’s Progressive, SH, Proportional, and random selection methods) under CD-CAT. Results shown that PWKL had the best precision on profile and individual attribute estimation, however, the index of exposure balance was very large more than the other methods. Except random method, the rest of methods had good recovery for individual attribute and efficiently suppress overexposure rate than PWKL. It seemed that trade-off relationship between precision and exposure balance.
Sharing, Question and Future study:
1) To my knowledge, iterative SH and Barrada’s progressive methods can’t maintain the exposure rate to meet the pre-specific maximum exposure rate. As such regards, the authors developed a restricted formula to control exposure rate as mentioned on page 261, “A way to suppress overexposure is to add a restriction so that the maximum exposure rate will be kept under a certain value, r.” It was really control the exposure rate to meet the pre-specific maximum exposure rate. However, it would lost the relationship of randomized and information while the left part of equation 10 is acting.