A Stochastic Method for Balancing Item Exposure Rates in Computerized Classification Tests
Issues of controlling item exposure rate have been investigated in computerized adaptive testing. There are many methods that have been developed and Sympson-Hetter (SH) method is the most popular one among these methods. It did not draw much attention for such issue in computerized classification testing. In the present study, the authors combined the method of stochastically curtailed SPRTs (SC-SPRTs) which was proposed by Finkelman in 2008 and SH method in computerized classification testing for maintaining identically well of percentage of correct classification with less average test length, decreasing the percentage of items which over maximum exposure rate, and increasing the percentage of items which less popular for being administered, comparing to SH only. A series of simulations were conducted with content balancing and three levels of gamma. Results indicated that the method of combining SC-SPRT and SH performed quite well for controlling item exposure issue.
Questions and comments:
1) Generally speaking, the relationship between accuracy (person measure) and exposure rate controlling is trade-off in computerized adaptive testing. However, such statement seems not exist in the present study. For computerized classification testing (CCT), the percentage of correct classification (PCC) can be referred to be an index of accuracy. I think the main reason is that SC-SPRT itself can avoid administering those useless items, thus average of test length (efficiency) can be efficiently decreased. As a result, some indices about exposure controlling can be improved. I think the paper shed more light on the conventional SPRT wasting too many items rather than issue of exposure controlling due to the fact that the method did not control exposure rate to fulfill pre-specific rate of 0.2 for all items.