This study compared three reliability coefficients, including ρ, ω and π. ω is used for continuous data and based on the sum raw scores. ρ is another measure of reliability, which is based on the maximum likehood factor score, taking into account the fact that people with the same sum score can have completely different response patterns across items. π is introduced as reliability for the dichotomous IRT models, using the information function.
These three kinds of reliability coefficients were compared mathematically and used the simulation results as evidence. Results showed that the ρ is always the maximal reliability, as it takes into account all the information the response patterns carried. ω will be lower than ρ as it use the unweighted sum raw scores, which ignore different response patterns under the same sum score. π is reasonably even lower as it dichotomized continuous responses, which further loss information an original response pattern could have, however, the impact could be reduced by adding more response categories. π can be greater than ω if the discrimination parameters vary a lot among items.