Psychological and educational measures were often developed to assess multiple dimensions of a construct. Such data are usually assumed to be unidimensional within individual subscale and item parameters of each unidimensional subtest are estimated separately in unidimensional IRT approaches. However, such approaches ignoring relations between domains might result in underestimated reliability or biased parameter estimates. Researchers have advocated using multidimensional IRT (MIRT) approaches to obtain more accurate estimates of parameters by taking into account correlations between domains. This paper aimed to examine with the MMLE method whether or not the MIRT performed better than unidimensional IRT approach did using the accuracy of item parameter estimates as a criteria when a test has a single structure (study 1) or a mixed structure (study 2).
Results in study 1 indicated that when the MMLE method was applied and the number of items was small, the MIRT approach yielded more accurate estimates of parameters whereas the number of items was large, the unidimensional IRT approach performed better if the simple structure assumption was tenable. Results in study 2 showed that when a test had a mixed structure but was treated as single-structure, the correlations between subscales were overestimated and the ARMSE of the estimated IRFs was large.
I agree with the author’s opinion that “one must examine the simple structure assumption before proceeding statistical analyses based on the assumption“(p.396). It is necessary to check assumptions before any data analysis methods are conducted. But I was surprised by the findings from study one. I was thinking that the MIRT approach would yield more accurate estimates compared to the unidimensional IRT approach as a correlation coefficient between to subscales increased. I have not figured out why the JMLE and MMLE yielded different conclusions.