Formal Relations and an Empirical Comparison among
the Bi-Factor, the Testlet, and a Second-Order
Multidimensional IRT Model
Frank Rijmen
Educational Testing Service
Journal of Educational Measurement Fall 2010, Vol. 47, No. 3, pp. 361–372
The bi-factor model (Gibbons & Hedeker, 1992), the testlet model (Bradlow et al., 1999; Wainer, Bradlow, & Wang, 2007), and a second-order model are described in the paper. It is also shown how the latter two are formally equivalent, and can be formulated as restricted bi-factor models.
The bi-factor model: Each item is an indicator of a general dimension and one of K other dimensions. The general dimension stands for the latent variable of central interest (e.g., reading ability), whereas the K other dimensions are incorporated to take into account additional dependencies between items belonging to the same cluster. An efficient EM algorithm can be used to obtain full-information maximum likelihood estimates for the model parameters (Gibbons & Hedeker, 1992), which is based on the complete joint contingency table of all items, rather than on the marginal tables up to the fourth order as is the case for limited-information estimation (Mislevy, 1985).
The testlet model (Bradlow et al., 1999;Wainer et al., 2007) : It is a special case of the bi-factor model, which is obtained by constraining the loadings on the specific dimension to be proportional to the loadings on the general dimension within each testlet (Liet al., 2006; Rijmen, 2009).
The second-order model: The second-order multidimensional IRT model for testlets incorporates a specific dimension for each testlet. It also contains a general dimension, but, unlike in the bi-factor and testlet models, items do not directly depend on this general dimension. Rather, items only directly depend on their respective specific dimensions, which in turn depend on the general dimension.It is assumed that the specific dimensions are conditionally independent: all associations between the specific dimensions are assumed to be taken into account by the general dimension. It is assumed that all the dependencies between the specific dimensions are accounted for through the general dimension.
Relations：
It follows that the second-order model is a restricted bi-factor model, where within each testlet the loadings on the specific dimensions are proportional to the loadings on the general dimension. These restrictions are the same as the restrictions on the bi-factor model to obtain the testlet model. The testlet model and the second- order model, are formally equivalent, and are a restricted version of the bi-factor model.
An International English Assessment Test was used for comparison of a unidimensional two-parameter logistic model, a second-order model incorporating a first-order dimension for each testlet, and a bi-factor model with a specific dimension for each testlet. The parameters were estimated with the eficient EM algorithm. The results show that the bi-factor model is the model to be selected.
The paper offers us an theoretical perspective to understand the relations between the three models.
1 The analysis of the real data does not include the testlet model, which makes the article incomplete.
2 there are no results of the loading of general and special dimensions, which is important for researchers to compare.