The effect of ignoring classroom-level variance in estimating the generalizability of school mean scores
Data for the annual measurable objectives of No Child Left Behind (NCLB) Act is usually analyzed by working model of school-level solely and ignoring class-level. The model assumes that variance of class is 0, however, it would not always meet the assumption in reality. The present paper conducted design of fixed and random effect of classroom-level variance and to examine the performance of G coefficient and SE when data is analyzed by conventional model. Results pointed that the higher the number of classes and students per class, the higher the G coefficient would be. Comparing with random effect on class, ignoring one would tend to overestimated G coefficient; on the contrary, it would be underestimate G coefficient for comparing with fixed effect on class variance. In terms of standard error for school means, the higher the number of classes and students per class, the lower the SE would be. For the true model of fixed effect on class variance, the ignoring one inclines to overestimate the SE slightly. However, for random effect on class variance, the ignoring one would severely underestimate the SE. In the end, analysis of real date demonstrated the similar consequence of previous conclusion.
Questions
1) How to manipulate ICCs and ICCc?
2) I’m a bit confuse on the difference of meanings between G coefficient (reliability) and SE.