Item Response Modeling of Forced-Choice Questionnaires
The authors successfully developed the IRT Thurstonian model for forced-choice and ipsative data format. The proposed model can effectively solve the problems that conventional rating model cannot do them which are distortion of scale relationships and comparison among individuals. Two simulations were conducted and several independent variables were considered: keyed direction of items, number of traits, correlations between traits, and block size. The first one simulation considered smallest two traits into study, and results illustrated that item parameters and their SEs could be estimated accurately when positively and negatively keyed items are balanced. Moreover, the correlation between two traits was also well recovery. However, conditions with all positively keyed items shown poor performance on test reliability. The second simulation investigated parameters recovery of five traits under different block size (pairs, triplets, or quads which is two, three, or four per block). It was found that 20 triplets and 15 quads can provide sufficient measurement precision for designs of triplets and quads, respectively. Finally, a real data was analyzed by the proposed model and it was shown that lower reliability estimates were obtained from the real data when comparing with the second simulation.
Questions
1) Can we compare the same trait between different individuals? It seems that we don’t have the same benchmark to do so due to the fact that the specific trait would be estimated by different amount of items belong to the same trait with one another.