The major advantage of computerized adaptive testing (CAT) is that it allows the test to home in on an examinee’s ability level in an interactive manner. The aim of cognitive diagnosis is to provide information about specific content areas in which an examinee needs help. This study proposes the combination of the estimation of individual ability levels (θ) with an emphasis on the diagnostic feedback provided by individual attribute vectors (α), thus kinking the current standard in testing technology with a new area of research whose aim is to help teachers and students benefit from the testing process. The technique utilizes the shadow-testing algorithm to simultaneously optimize the estimation of both the ability level θ and the attribute vectors α. There conditions for comparison: (1) θ-based condition, item selection was based on θ estimate only; (2) α-based condition, item selection was based on α estimate only; and (3) θ- and α-based condition, item selection was based on both estimates. The results show that, first, the θ- and α-based condition outperformed the α-based condition regarding θ estimation, attribute mastery pattern estimation, and item exposure control. Second, both the θ-based condition and the θ- and α-based condition performed similarly with regard to θ estimation, attribute mastery estimation, and item exposure control, but the θ- and α-based condition has an additional advantage of in that it uses the shadow test method, which allows the administrator to incorporate additional constraints in the item selection process, such as content balancing, item type constraints, and so forth, and also to select items on the basis of both the current θ and α estimates, which can be built on top of existing 3PL testing programs.
Questions:
1. Why the same data set can be calibrated with two different models (3PL and Fusion model were used in this study).
2. How to obtain or calculate α estimate in the θ-based condition? Similarly, how to obtain θ estimate in the α-based condition?