Three analyses were adopted to assess the presence of random guessing in MC items. The first analysis used the whole response pattern to calibrate item parameters, then compared with simulated item estimates, estimates of most items are overestimated and underestimated when difficulty with negative and positive values, respectively. The tailored analysis can be treated as main method that the author tried to promote in the present paper. The tailored analysis changed some response from the whole sample which are judged as random guessing, then calibrate item parameters from the rest subsample. The result shown that the tailored has much better item recovery, however, it has larger variance of estimates of parameters. The anchored analysis also had been done here for fixing the first six items.
Several comments I want to share is in the following:
1) The author mentioned that “with guessing by less proficient persons, the number of persons who answer an item correctly is greater than it would be without guessing” (page 6). As my point of view, the assumption is not strong enough. In the other word, I don’t agree with the statement that the author proclaimed here. In the situation of random guessing, the statement may be correct, however, the probability here should not be computed by using equation (8). The author can use c instead of p+c(1-p)y to illustrate the property of random guessing. However, once students have processed by his/her ability, it would not be a random guessing, then p+c(1-p) (i.e. the 3PLM) is not reasonable here. The author can also think about that guessing may be provoked by person ability.
2) The tailored analysis is a good way to calibrate item parameters instead of estimate person ability. It does not involve fairness of examination but can calibrate parameters much accurate.
Question:
1) Andersen theorem seems to play an important role from the title of paper, but I couldn’t get the main point due to English problem. Can anybody explain it more clearly?