Modeling Answer Changes on Test Items
Wim J. van der Linden
CTB/McGraw-Hill
Minjeong Jeon
University of California, Berkeley
This paper proposes a probability model that can distinguish between the probabilities of initial answers and subsequent erasures, and condition them on the test taker’s ability. The model can be used for an analysis of the patterns of residual WR erasures.They also introduce a statistical test derived from it with the number of Wrong to Right erasures as the test statistic. An empirical data set from a recent large-scale assessment is used for the detection procedures. The 3PL model is used to model the two stage process. They also present results from a study of the robustness of the model with respect to an assumption about the testing time on which it rests, and discuss how the procedure should be implemented when the assumption is violated. The procedure, which is available as the function bayesglm in the arm package for multilevel regression analysis in R, was used in the study.
Comments：
Is there some mistakes in Equation 2 ? Is it not the probability of an RW erasure but RR erasure? However there are some basic ideas deserves scrutiny. The first one is that ‘It does not make much sense to assume test takers guessing on any of the items during their review of their initial responses’, from my own experience, If I have time, I will go over the questions that I am not sure during the initial stage. So personally I will guess one answer, if I have time then I will go directly to the question I guess. And for the second assumption, if I have time to go over the items I am not sure, then it is still possible to guess if I can’t figure it out. So there are some problem solving strategies here for different people.