The author tried to include the variances within different levels (pupils, classes, and scools) into modeling by G theory. In Table 1 and Table 2, it shows that the variance within classes cannot be neglected. The pupils are usually treated random effects. Class effect can be treated as random or fixed. The reliability of school-level, c facet, or ignoring the class effect are presented in formula. Then the table 3 design different conditions on the values of parameters. Figure 1 shows that the reliability is high when class effect is fixed. More the fixed effect, more the reliability. In contrast, the random class effect is treated as random error. Thus, larger the variance, lower the reliability. Figure 2 shows the standard errors under different conditions. In empirical data, the results show that the G coefficient will be higher if class effect is treated fixed. Otherwise, it will be lower when class effect is treated as randomness.
Comments:
1. In p. 15, ‘split plot’ design means that the students nested in classes and same items are administered to all students?
2. In p. 15, it reads ‘IRT scaling accounts for form-to-form variation in item difficulty, there is no need to include item or form effects’. What the form-to-form variation means? Did the study use the IRT method to modeling?
3. Why need to be divided by (nc) and (npnc) in the denominator of formula (3)? And why the SE is formula (4)?
4. After reading the paper, I found that I’m not very familiar with G theory even I have read related papers before.