38 Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model (Present by Nicky)

Wayne's comments

Wayne's comments

by CHEN Chia Wen -
Number of replies: 0

The purpose of this paper is to estimate some of uncertainty skill in the Q matrix, and the result encourage us to use his approach very much. The Author introduced the Bayesian extensions of RDINA and HO-RDINA, which are the CDM models referred at the previous studies, for estimate the posterior of the elements of the Q matrix. Some of uncertainty elements treated as random variables stand on Bernoulli distribution. The prior as Beta distribution combines with the Bernoulli distribution. It yields the posteriors standing on Beta distribution also. The simulation study investigated the recovery rate of uncertainty elements by several factors: uncertainty elements completed in whole items, uncertainty elements completed in whole skills, certainty elements are misspecified, and HO-RDINA are generated. The results showed there is quiet good recovery while the certainty elements correctly specified. The uncertainty elements in whole skills are affected by misspecification very much, whereas the uncertainty elements in whole items just are bad a little. Generally, the recovery of uncertainty elements are excellent. It suggested this approach could lead an better accurate detection.

1) In Condition 6, the posteriors tended to be around 0.5. It lead the uniformly poor results. I think this phenomenon can be solved by the methods mentioned by reviewer, that setting the certain cut points and running multi-stage to detect each element step by step.

2) In Table 4, the skill 4 in item 1 performed less always. I am curious the cause of it. I guess it perhaps resulted from the item 1 just with one skill including. For investigating this question, the item 1 ~4 should be set the uncertainty elements separately. The result of item 2~4 would be the same with item 1 if the inference was correct.

3) the simulation of HO-RDINA set the discriminate parameter as a constant value 3. If we just set the discriminate parameter equal to 1, how would the effect of recovery be affected?