In cognitively diagnostic measurement methods, It is necessary to establish Q-matrix for representing accurate specification of knowledge and skills. The Q-matrix is making by experts' subjective determining. It is difficult to avoid the Q-matrix we used is not true Q-matrix. This study investigate the consequence of this misspecification. two types of misspecification investigated in this paper was excluding an essential attribute and including a superfluous attribute. The diagnostic method this paper used was rule space method. In addition, the author also examined the effects of order relations among attributes. The order relations can be explained as the higher cognitive concept was learned after learning the lower cognitive concept. It implicate that if examinee has higher order(subset) attribute, he or she must have lower order(superset) attribute. Likewise, The lower order attribute failing means the higher order attribute failing absolutely.
The result shows five point mainly. First, the consistencies between true attribute and estimated attribute was lower for excluding attribute than including attribute.
Second, AMP would underestimate examinee when misspecification appear as fallowing: 1) exclusion of an essential attribute, 2) excluding superset attributes, and 3) including subset attributes. In contrast with these situations, fallowing would lead AMP overestimating examinee: 1) inclusion of a superfluous attribute, 2) including superset attributes, and 3) excluding subset attributes.
In my comment, I am interesting in the order relations. Such as what the author said in the implications, a test designer makes insufficient items would cause the order relations appearing. I think when Q-matrix has made, we should check if there is any pair of attributes with a order relation. If yes, we should present this relation to experts and ask them ensure this relation is true. Otherwise, the test designer has to make more items for eliminating this relation.