Recognizing Uncertainty in the Q-Matrix via a Bayesian
Extension of the DINA Model
Lawrence T. DeCarlo
Background：
The goal in cognitive diagnosis, as the name suggests, is to ‘‘diagnose’’ which skills examinees have or do not have. The use of a CDM requires the specification of a Q-matrix. The Q matrix is usually determined by expert judgment, and so there can be uncertainty about some of its elements. The Bayesian extension of the DINA can be used to explore the uncertainty.
Firstly, the DINA model and its reparameterization are presenting with their formulas. Then the Bayesian extensions of the models are described which allows some of the elements of the Q matrix to be random instead of fixing all the elements to 0 or 1. The posterior distributions are used to guide decisions about the element in question.
8 simulation studies were conducted for demonstrating the model. The data were simulated according to some real data analysis. Simulations show that this approach helps to recover the true Q-matrix when there is uncertainty about some elements.
The model is very interesting which seems can solve the practical Q-matrix problem, however the premise is that first there must be some correct elements in the Q matrix which is defined by the expert, base on this hypothesis，the model then can be used for detecting the uncertainty and misspecification. The uncertainty of the matrix will lead to less accurate detection. The misspecification will affect some of the recovery of the uncertainty not all which seems no rule to summary. It there some criterion we can use to just the model can be used or not according to the percentages of the recovery?
Is there some way to measure the validity or reliability of the Q Matrix?
The thinking aloud experiment can be used to provide evidence for the model’s utility.