ARC Laboratory Sharing

A New Stopping Rule for Computerized Adaptive Testing

by Seung W. Choi, Matthew W. Grady and Barbara G. Dodd (20 10 ) .

Two most commonly used methods for determining computerized adaptive testing (CAT) are the fixed length and variable length stopping rules. The primary advantage of the fixed length stopping rule is its simplicity, that is every examinee administered the same number of items can be terminated the test; but it causes different degrees of precision for examinees. In the contrast, variable length stopping rules seek to achieve a certain degree of precision for all examinees. Two commonly used types of variable length stopping rules were the standard error (SE) and the minimum information stopping rules. For the SE stopping rule, the test terminates when a predetermined SE is reached. The disadvantages of the SE stopping rule are that it may limit the efficiency of a CAT by unnecessarily administering additional items to examinees for which a predetermined SE criterion cannot be meet; and it may limit measurement precision by terminating the CAT, even though informative items are still available for administration. The rationale of the minimum information stopping rule is that a CAT is terminated when there are no more available items capable of providing a predetermined minimum level of information for an examinee. Compared to the SE stopping rule, the minimum information stopping rule has an advantage is that it tends against the needless administration of items to examinees for which a high degree of measurement precision is not possible; and a disadvantage is deliver less accurate measurement precision than the minimum SE stopping rule.

Accuracy and efficiency are two major purposes for CAT whether the stopping rule is fixed length or variable length. However, there is a trade-off between measurement precision and testing efficiency because two potential problems. First, when a CAT is stopped, even though desired gains in information would result from the administration of additional items. Second, when items are unnecessarily administered to examinees for which more precise trait estimates are unlikely. The second problem can be especially problematic when the item pool information and the examinee trait distribution are not matched. Based on the aforementioned problems, this paper proposed a new stopping rule to balance measurement precision and testing efficiency simultaneously, called the predicted standard error reduction (PSER).

The steps of the PSER stopping are: first, when the minimum SE stopping rule is reached, if the PSER reduce by a predetermined level then continue to test; that is it can improve measurement precision by administered additional one or two informative items. Second, when the minimum SE stopping is unreached, if PSER reduce by at least a predetermined level then stop to test; that is it can reduce administered unnecessarily items when a high measurement precision is not possible. The performance of the PSER stopping rule was evaluated by a series of simulations. The results show that the PSER makes efficient use of CAT item pools, administering fewer items when predictive gains in information are small and increasing measurement precision when information is abundant.

Question: