The HRM-SDT combined the HRM and SDT to model the rater’s severity or central tendency. It avoided the highly reliable rater problem and capable of rater’s perception for essay. The most complicated part is the SDT in this article. In addition, its parameter estimation required skillful methods to solve. The most interesting part is it model each rater’s severity and perception for an essay. These plus parameters aid to explain each rater’s scoring behavior. Simulation studies (omitted) showed parameters can be recovered well while essaies or items are enough. Finally, an empirical data was fit to show HRM-SDT provided us more information about rater’s severity or scoring tendency.
1. I can’t figure out why the higher value of Ф means severe and vice versa. We know the function (1) is a symmetrical function. No matter Ф is getting higher or lower, it will decrease the probability. How can we tell the meaning of Ф?
2. Figure 2 and function (2) are hard to be related together. I cannot immediately realize their relationship.
3. in figure 6, how do we calculate the four horizontal lines between 0 and 1?
4. In figure 6, it means the rater likes to give higher score if circle is located below its baseline. It is hard to relate the concept to figure 2. But it seems reasonable when we read function (2).
5. It is curious how is their model-data fit? (HRM and HRM-SDT) Would HRM-SDT fit better? Although HRM-SDT have more parameters to be estimated and explain.
6. It is interested to combine IRT model and SDT. Figure 6 uses threshold to explain the rater’s severity, but it is not quite intuitive as in IRT-rater model. If we can use IRT approach at all, it will be easy to explain the results.