Can Subject Matter Experts’ Ratings of Statement Extremity Be Used to Streamline the Development of Unidimensional Pairwise Preference Scales?
Stephen Stark, Oleksandr S. Chernyshenko, and Nigel Guenole
The empirical and simulation studies were conducted to examine whether subject matter expert (SME) ratings of statement extremity (location) can be substituted for MML estimates to streamline test development and launch:
The author firstly used one study to examine the correspondence between SME and MML location estimates for two UPP personality scales and the resulting effects on trait scores in a large sample of examinees. Then they conducted a Monte Carlo simulation to examine the recovery of known trait scores when responses were generated using ‘‘true’’ item parameters, but scoring and adaptive item selection were based on simulated SME location estimates having varying degrees of error.
Comments:
The model can be used to ipsative test
If the UPP means all the items involving statements should represent the same dimension for one questionnaire, then the first step we should do is to make sure the questionnaire is uni-dimensional, which from the other perspective, it means all the method here cannot generalize to the multidimensional tests.
The author said, the slope depends on the distance between the statements composing an item In formulation. However in formulation 2, a=(2q-(us+ut)), formulation 3,b= us-ut. So according to this, b here means the slope parameter?