Comparison of Reliability Measures under Factor Analysis and Item Response Theory
Reliability is a core concept in psychometric filed and it has been widely and simultaneously used on conventional and modern measurement, such as alpha for classical test theory, omega and rho for factor-analysis measurement, and pi for item response theory measurement. The relation of reliability coefficients of classic one and factor-analysis one was well studied. However, the relation of reliability coefficients of factor-analysis one measurement and IRT based one is not yet investigated before. Therefore, this paper was proposed to study such relationship aforementioned. At first, the authors illustrated the mathematics relation of rho and pi(2) of dichotomous data. Also, pi(2) would smaller that omega in practical conditions thought no mathematics relation cannot be formed. Other results shown that pi would surpass omega if the number of categories keep increasing.
Questions:
1) It seems that this paper focus on the relation of different reliability coefficients solely. How do we choose which reliability coefficients between omega and rho to report in practice?
2) In IRT, reliability involves the number of categories. Does it mean that we should choose polytomous items to administer if we would like to obtain high reliability?