Statistical Consequences of Attribute Misspecification in the Rule Space Method
Q-matrix is an important part for cognitive diagnosis assessment. Once Q-matrix is not well specified by experts, it would cause incorrect report for examinees. Two researches have been investigated for aforementioned issue in the past. These two studies were based on the linear logistic test model and the DINA model, respectively. Likewise, this paper investigated the issue of attribute misspecification in the rule space method. A series of simulations were conducted with main manipulated variable of misspecification, excluding on essential attribute and including a superfluous attribute. For the condition of excluding on essential attribute, it was found that some knowledge states were eliminated due to attribute exclusion. As a result, classification rate would be decreased and the AMPs of the examinees were based on limited knowledge states, leading to higher error. On the other hand, more knowledge of states would be created for the case of including a superfluous attribute. Therefore, classification would be slightly affected by redundant knowledge of states.
Comments, Questions and Future Study
1) The paper clearly depicted statistical consequences of incomplete Q-matrix in the rule space method. It would much better that we should find a method to improve the misspecification and biased estimation in a positive attitude if Q-matrix of misspecification is not avoided.