Question: What is Thurstonian IRT doing?
My answer: The ipsative data which comes from the classical scoring of the forced-choice questionnaire cause some problems for estimating latent traits since the residual error variance of the latent response variable y* are zero. That’s why Thurstonian factor model which could be used for comparative choice is not applicable to it.
The Thurstonian IRT model was reparameterizing Thurstonian factor model, expressing the second-order factor model for binary data to the first order model equivalently. “ The whole point was to recover normative traits from forced-choice ratings. The scores recovered with the Thurstonian IRT model are no longer ipsative, i.e. centering of scores on the person average disappears. ”(Communicate with Anna).
Question: Suppose two interviewers for a job have identical score in responding to the personlaity test which are forced-chocie items, can the model in Thurstonian IRT model identify the quantitative difference between these two participants?
My answer:
No.
First of all, we should pay attention to the difference between "the same ipsative score" and "the same response pattern". The ipsative scores is the rank-order of the statements.The response are coded as binary outcomes of pairwise comparison within the block and the pattern is the pattern of choices in the block. Competely different patterns may have the same ipsative score.
In non-epsative test, such as academic performance exams , when students have the identical raw score, they can be LOGICALLY treated as having the same ability levels . If they have different latent trait, the raw score would be different. The difference between the two persons could be compared because the same scale is used for the comparison.
But, for forced-choice questionnaires, the responses come from ipsative comparison. Hence, two persons who have very different latent traits may have the same selection of the statements that is ‘most like me’ or ‘least like me’, that lead to the same ipsative score.
The problem is fundamental. However, it is ignored or we may even say could not solved in the Thurstonian IRT model because it was assumed that the individuals with the same responses pattern will get the same score.
So, the test design was very important so as to ensure two persons with different trait levels must have different response patterns. The good and bad designs were illustrated in the simulations and are discussed in the conclusion.
Future study:
Models in the present study consider only dominance items are used. Future study could model forced-choice tests with ideal point items, as the article suggested.