Statistical Consequences of Attribute Misspecification in the Rule Space Method
Seongah Im and James E. Corter
Educational and Psychological Measurement 2011 71: 712
The present study investigates the statistical consequences of attribute misspecification in the rule space method for cognitively diagnostic measurement.
Two types of attribute misspecifications:
l exclusion of an essential attribute
l inclusion of a superfluous attribute
A simulation study was used to illustrate.
The results show that if the first misspecification tends to lead to underestimation of examinees mastery probabilities for the remaining attributes, whereas the second misspecification leads to overestimation of attribute mastery probabilities for the other attributes.
The paper reminds us the importance of constructing a Q matrix before we analyze the data in CDM. Since in the simulation, we know the students’ knowledge states, then it is easy for us to judge the result. However, in the real condition, we do not know the Q matrix is completes or misspecification, as the author mentioned that mapping of knowledge states to ideal response patterns is not a one-to-one correspondence, so how can we make sure the students’ knowledge states ? In my opinion, there should be some techniques to evaluate the Q matrix or some parameters involved in the study to make sure the Q matrix is right, so that we can avoid the underestimation or overestimation of the students’ knowledge state.
The second problem is that the consequences of attribute misspecification could be examined only for examinees successfully classified into any of the knowledge states, because those who were not assigned into any states could not be further analyzed in the RSM. What can we do about these examinees?