A New Stopping Rule for Computerized Adaptive Testing
This paper applied a new stopping rule for computerized adaptive testing on medical and psychiatric assessment. The new stopping rule (PSER) is based on the posterior variance, which determines two minimum expected reductions for hyper (0.03) and hypo (0.01) parameters. Once the expected reduction is larger than 0.03, an item will be administered; on the contrary, the expected reduction is less than 0.01, CAT will be terminated. The PSER can improve measurement precision for those who are in targeted group and decrease the burden for those who fall outside the targeted group. These dual considerations on measurement precision and test efficiency can be taken into account on the PSER for medical and psychiatric CAT. The authors conducted a series of simulations of the PSER and the other two existing stopping rules (the minimum SE and the modified minimum information). The results illustrated that some properties of the PSER on number of items administered and RMSE, and that compared with the minimum SE and the modified minimum information stopping rules. The prevalence of Hyper and Hypo cases were also discussed on the final part of paper.
Sharing, Question and Future study:
1) It’s useful application of the PSER on the medical and psychiatric assessment than the other conventional stopping rules. In medical and psychiatric assessment, patients would not argue that why someone took much more or less items than them. The main concern for them is how much more accurate decision can be made if they are targeted patients, or decreasing of needless items be administered if they are not patients. And the PSER can meet the aforementioned goal. However, it would involve fairness problems in ability assessment result from the method is not based on fixed length or equally SE standard.
2) The original minimum information stopping rule would also has hyper and hypo cases, I think it would much more reasonable that the authors should include it into simulation. If the performance of the method is as well as the PSER, we don’t need to use a more complex method of the PSER.
3) How to set the hyper and hypo parameters? What’s the rationale of 0.03 for hyper parameter and 0.01 for hypo parameter in the present study?