ARC Laboratory Sharing

This paper investigated the effect of misspecification of Q-matrix on parameter estimation and participant classification for the DINA model. The design of misspecification of Q-matrix in this study is very interesting.

Misspecification of Q-matrix can be grouped into two sets:

1. Incorrect specification of whether an item measuring an attribute.

a. under-fitting: specifying 0 where there should be 1

b. over-fitting: specifying 1 where there should be 0

c. balanced misfit: exchanging 0 and 1 while controlling for the overall number of changes.

2. Incorrect dependency assumptions about two attributes.

In simulation, 4 attributes design was used, measuring by 15 items, so each item testing one unique attribute pattern. According to the definition of misspecification previously, the whole design consists of total 12 conditions, as follows:

Results were summarized based on two parts, the item parameter and the person classification.

For item parameter, a) under-fitting conditions, over-estimate the slipping parameter; b) over-fitting conditions, over-estimate the guessing parameter; c) balanced misfit, an increase mean absolute deviation for both slipping and guessing.

For respondent classification, what is important is the effects on the space of all attribute patterns that were represented by the items through the Q-matrix.

This study covered kinds of misspecification of Q-matrix for DINA model and evaluated its effects. The design could be extended to other CDM models. Results showed that unusual large slipping or guessing suggesting misspecification of Q-matrix, however, there was no illustration of how large is “large”, and there was not a statistical testing for “large”. For practical use, the real parameter is unknown, it will be hard to determine whether an estimated parameter is large or not.