Robust Estimation of Latent Ability in Item Response Models
Christof Schuster and Ke-Hai Yuan
JOURNAL OF EDUCATIONAL AND BEHAVIORAL STATISTICS 2011
Unexpected response patterns caused by guessing, cheating, or carelessness may not be inferred correctly from the likelihood function of a simple standard model. In the former research, the usual strategy to handle unexpected response patterns is to eliminate inconsistent item responses. For several reasons, this is not a desirable procedure. An alternative strategy is to decrease their influence on the ability estimate, which can be accomplished by modification of the maximum-likelihood estimation equations, such that less weight is given to observations that are more prone to response disturbances. Mislevy and Bock (1982) suggested to ‘‘robustify’’ ability estimation based on Tukey’s bisquare or biweight function, which is implemented in BILOG. However its estimates are prone to head toward negative infinity for unusual response patterns in which correct answers are sparse.
In this article, an alternative approach to robustify ability estimates is proposed, that does not appear to suffer from such convergence problems. The approach replaces the bisquare weighting function with Huber weighting function. The three ability estimators—maximum-likelihood, bisquare, and Huber—are compared by computer simulations
Regarding bias, the new estimator can be considered a compromise between the bisquare and the maximum-likelihood estimator. If reduced bias is preferred over reduced sampling variability, the bisquare estimate should be selected. In the reverse case, if reduced sampling variability is preferred over reduced bias, the Huber-type estimate should be preferred.
There is no absolute method to resolve all the problems. Every method has its own limitations and advantages. The real data can be used to do this comparison.