This study established the relationship between an item IRT-based reliability coefficient (π(2)) and two factor-analysis-based reliability coefficients (ρ and ω). The relationship is build upon the equivalence between the unidimensional normal ogive IRT models and the unifactor models. They are analytically show that the relationships are ρ≥ω and ρ>π(2). The reasons why π(2) and ω fall short of ρ form an information gain/loss perspective . On π(2) ‘s part, the information loss comes from dichotomizing continues responses; on ω’s part, from using unweighted sum score (observed score = True score + error), which ignores the fact that response patterns may still differ given the same sum score. The simulation results have shown that, first, ρ always larger than the two others. Second, there is no dominant relationship between π(2) and ω. Third, as score categories (g) increases to infinite, IRT-based reliability coefficient finally converges to ρ; that is getting the continuous response back.
Comments & Questions:
1. Figure 2 and 3 were demonstrating that as score categories increases, IRT-based reliability coefficient will catch up with ω. But there are three horizontal lines and they are represented ρ (blue line), ω (red line), and π (green line) based on specified as Figure 1. Why plot these three lines?