The author proposed an approach to specify items on uncertain skills (attributes) in a Bayesian framework. To simplify the notation of the DINA model in Bayesian inference, the reparameterized DINA model was given first. Then eight conditions were investigated in simulation studies to take into account the proportion of uncertainty, the impact of misspecification of items on skills, and the uncertainty of skills involved. The results shown that the uncertainty of skills involved have serious impact on the proposed approach, while the approach is generally well in other conditions. To illustrate the application of the approach, an empirical data set was analysis.
1. The Q-matrix is involving two aspects: skills (attributes) involved and the specification of items on skills. This paper demonstrated a good utility for determining the specification of items on skills on some conditions. However, for the aspect of skills involved, it failed to take care the skills (condition 5 & 6). In reality, the two aspects of constructing Q-matrix is equally important and are asked for further study, especially for the skills involved.
2. Since the design of simulation study is not systematic, the magnitude of uncertainty of elements in Q-matrix, as well as the distribution of attributes on items and the presence of misspecified elements, is needed further study to answer the question that how powerful the proposed approach to offer information about specification of items on skills.
3. Can the similar approach be applied to the determination of item-level dimensionality in the IRT framework? The idea is very similar.
4. Just for curiosity, how are the impacts if one or more skills (attributes) ignored when constructing a Q-matrix?