To address the problem of construct shift that might occur in item response theory vertical scaling, the authors proposed a bi-factor model to monitor construct drift at each assessing time point and to model the common latent ability, simutaneously. Based on the work of Gibbons and Hedeker (1992), the proposed model was assumed that the grade factors are orthogonal to one another and to the general factor, and the general factor across all time points is maximizely extracted at individual grade levels. A common-item design was used to examine the performance of a bi-factor model in vertical scaling with concurrent calibration. Results from simulated and real data indicated that a bi-factor model, compared to unidimensional Rasch model, in vertical scaling with construct shift yielded more accurate estimates of person parameters, item discrimination parameters, and group mean parameters.
Comments:
(1) Construct drift might imply two things. First, a test might measure different constructs at different time points. For example, at an earlier stage, it might measure “oranges”, but assess “apples” at a later stage, although we can say this assessment is measuring “fruit”. Therefore, how to transform the results to the same scale for comparisons becomes much difficult. Or the scale might be multidimensional, and the item difficulty hierarchy of common items will change across time. As a result, unidimensional IRT model won't fit the data well, and a bi-factor model might underestimate person parameters.
(2) If the previous problems do not exist, it will be interesting to compare the performance of a bi-factor model in different linking designs. Previous studies have indicated that various linking designs might tell very different stories about person ability changes. If the proposed model yields very similar findings regardless of linking designs, it will be very meaningful for longitudinal studies.