How should we assess the fit of Rasch-type models?
Approximating the power of goodness-of-fit statistics in categorical data analysis
Maydeu-Olivares A.
Montano R.
In this paper, the asymptotic power of two statistics specifically designed to assess the fit of Rasch-type models (R1 & R2) and a general statistic (M2) were compared. The authors hoped to answer a question: whether statistics specifically designed are needed, or whether general statistic is suffice to test goodness-of-fit. The results showed that the general statistic (M2) had a higher power in again 2PL, 3PL and multidimensional 1PL model when the slop parameters were high. Thus, it was concluded that ‘there is no clear advantage in using statistics specifically designed for Rasch-type models to test their goodness-of-fit.
Questions and Comments:
1. It was found in simulations that M2 had higher power than R2 in again 2PL, 3PL, and multidimensional model. However, in two empirical examples, R2 was found had higher power. I think additional simulations/analysis are needed: (1) in the present simulation, only sparse situations were simulated. But, in the first empirical example, the data was not sparse. It was not clear whether R2 will have a higher power than M2 when the data was not sparse. (2) for the second empirical example, the data was sparse. However, only 1PL was fit to this sparse data. It was not clear whether R2 had a higher power is due to the high slope parameters.
2. In last page, row 5, it is said that ‘R2 is most powerful’. I think it is a mistake. In simulation, M2 always had a higher power than R2, though the power is not high when the slope parameters are not high (a=0.5).
3. Only the overall violation was simulated in the present paper, no wonder M2 will perform better than R1 and R2. The statistic specifically designed is expected to show good power when the target assumptions were violated (such as local dependence).
4. Unless the item have high slope parameters which suggest enough information, the performance of the M2 are acceptable.