This paper examined if explained common variance (ECV) and coefficient omega hierarchical (omegaH) as well as popular fit indices (root mean square error of approximation, comparative fit index, and standardized root mean square residual) in SEM were informative to detect multidimensionality of a measure. The findings suggested that ECV and percentage of uncontained correlations (PUC) were significant indicators for multidimensionality when the true model was a bi-factor model. The three fit indices failed to detect the wrong model and to predict the magnitude of structural coefficient bias.
Comments:
1) Is a sum of correlation matrix equivalent to variance of observed scores on all items (page 11)? McDonald (1999) used variance(X) as the denominator but not a sum of correlation matrix.
2) In SEM literature, some estimation methods have been developed for data violating the assumption of multivariate normality. This paper did not mention if those approaches were implemented in data analyses, and what type of data they generated (dichotomous, categorical or continuous). The values of fit indices might be different from the ones in this paper. It will be great to know these before it is concluded that those fit indices failed to detect a true model.
3) In accordance with the findings in this paper, does it imply that it will be better to examine the measurement part in IRT first, instead of using a CFA approach?