Using Logistic Approximations of Marginal Trace Lines to Develop Short Assessment
In multidimensional bifactor model, the slope on theta1 indicates the relation between an item response with theta1 conditional on theta2, rather than the ‘pure’ relation between the response and theta. The confusion was clarified in Ip (2010) and a marginalized version of testlet model was introduced to compare the testlet response model and the standard item response model. As shown in equation (4), a unidimensional-equivalent marginal trace line for primary dimension can be derived by integrating the second-dimension out. The slope parameters and the threshold of the marginal trace lines can be compared to the multidimensional bifactor model through transformation using logistic link functions. In this study, the procedure was used to create unidimensional scales from multidimensional scale by means of a ECV statistic.
Questions and Comments:
(1) Though the marginal trace lines were nearly equivalent to the multidimensional bifactor model, it is hardly acceptable that the newly created short scale is unidimensional. Firstly, the items, as shown in table 4, do not seem to be unidimensional. Secondly, it was not shown how the authors can claim that the new short scale is unidimensional. Did the authors fit the data of the new short scale to a unidimensional model and found there is a good fit?
(2) Again, it was found only a single item should be retained per cluster the content of the particular secondary dimension, since the local independence will be violated if two or more items were selected from the same secondary dimension. However, will the estimation be accurate when only one item was used for one factor?
(3) The original purpose of Ip (2010) was to compare and interpret 2-PL and 3-PL model that are comparable to standard item response model. The application of the method to develop unidimensional an short test from the multidimensional data, though creative, is not acceptable. Part of reasons are mentioned in (1) and (2).