Just as in DIF study, the nonparametric approach is appeal because of its easy implement, less time and energy consuming. However, the implication is certainly restrictive and less precise.
In this study, a nonparametric approach for cognitive diagnostic which relied on classification by minimizing a distance between observed response and the ideal response for a given attribute is straightforward. The efficiency of the approach is examined when various DINA models (including DINA, DINO, GNIDA) are used for data generation and the Q-metrix is misspecified, in terms of agreement rate between the nonparametric approach and the parametric approach.
Questions & Comments:
1. In brief, the nonparametric is restrictive to low slipping and guessing parameters when data are generated from the DINA model. But, it is less tolerant of larger slipping and guessing parameters in NIDA data in which the slipping and guessing parameters can have a multiplicative effect. In addition, the approach does not perform well when a misspecifed Q-metrix is used. Hence, the limitations of nonparametric are obvious.
2. The classification rate of nonparametric approach depend on whether correct link between examinees’ attribute patterns and the given item skill patterns is specified (ie, conjunctive or disjunctive) for DINO data. However, in practice, the item patterns may consist of uncertainty. Hence, it is hard to specify the link correctly.
3. In nonparmatric approch, the number of ideal response will increase dramatically when the number of attributes is increasing. And large number of attribute may lead to problematic resuts. For example, in the empirical analysis, it was found that the classification agreement rate is only 45% between the nonparametric and parametric approach when there are 8 attribute (=256 patterns). How the nonparametric approch perfom if the number of attributes are larger?