A Multilevel Testlet Model for Dual Local Dependence
Hong Jiao
Akihito Kamata
Shudong Wang
Ying Jin
Local independence is one of the important assumptions for traditional IRT models.This assumption implies both local item independence and local person independence.This study proposes a four-level IRT model from the multilevel measurement modeling framework to simultaneously model both item and person dependence. The first level models item effects and the second level models testlet effects. The items are nested within the testlets. Level three models the effects of persons who are fully crossed with testlets and ultimately with items. The fourth level models the examinee group effects.
The simulation study and a real data were used to demonstrate the performance of the new model.The simulation results demonstrated that for both item and person parameter estimation, the bias was not affected but the SE was affected. The local person dependence factor had an impact on the group variance recovery: As the magnitude of local person dependence increased, the SE increased. The local item dependence factor significantly affected the SE and the RMSE in the testlet variance estimation with large effect sizes. The increase in the magnitude of local item dependence also increased the SE and the RMSE in testlet variance estimation.
Comments:
The logic of the paper is very clear. As we can see, many plots and tables are not provided in this paper which we can’t judge by our own except reading the summary information.
After all, the results of the proposed model has some difference from the testlet model in item difficulty parameter estimation and from the multilevel model in ability parameter estimation, considering the real situation especially the large scale test using matrix design, it should have some expansion in the future including improving the parameters and software.