65 Using the Many-Faceted Rasch Model to Evaluate Standard Setting Judgments (Present by Snow on 27May 2013)

Wayne's comments

Wayne's comments

by CHEN Chia Wen -
Number of replies: 0

1) I think the cut score parameter should be explained carefully. The logit of cut score cannot be read as cut scores directly. We should transform the probability of correctly answering by cut score of each category to the expected scores which indicate the cut score of each category.

2) In Figure 3, the horizontal axis is the item difficulties estimated from the panelist's evaluation data and vertical axis is the probability of item corrected gotten by student. I am curious that why not the author replaced the probability to the item difficulty estimated from the data implemented by students. Since the probability data shouldn't be fitted by linear model, the plot for comparing the difficulty parameter from standard setting and from students' response is more appropriate to fit linear model.

3) In figure 2, the residual for panelist 6 in the first round seems to be not random, a little negative slop, and the residual for panelist 8 in the first round was a little positive slop. This means probably there is the order effect on the items evaluated by panelist, that is, panelist 6 might be more and more lenient under the progress of evaluation but panelist 8 were more and more server.