35 Characterizing sources of uncertainty in item response theory scale scores (Present by Sherry)

cw's

cw's

by LIU CHEN WEI -
Number of replies: 0

The paper aims to correct the standard errors of measurement (SEM) in multiple imputation way. It is shown that the SEM for EAP or MAP estimate is often underestimated and the scale’s marginal reliability overestimated in past researches. The imputation method is straightforward and easily implemented, provided that the asymptotic covariance matrix of the item parameter estimates is available. First off, it uses the supplemented EM to obtain the covariance matrix of item parameters; secondly, the MI method is used to infer the EAP scores which has considered the uncertainty of item parameters. The MI values are drawn from multivariate normal distribution. The main feature is that the author incorporated the MI into the modified EAP and the estimation of variance. And the confidence envelopes is also available by using MI method. Finally, an empirical data and simulation studies were conducted to investigate its performance on the estimate of latent trait and item parameters. Although the uncertainty of estimation of item parameter estimation is small, it is still possible making big effect when sample size and the number of items is small.

1. Although EM is conceptually straightforward, but it is a laborious process in coding and no existing software can do it for specific model. The MCMC is most remarkable method for parameter estimation in present. However, limited to the lack of high-efficient personal computers, many flexible EM methods for specific model have been developed. When it comes to the uncertainty of item parameters, the MCMC has included them at all via one-stage estimation. It is curious for me to know if the MI can actually do better in estimation than the MCMC. Further comparison between them should be considered as well.

2. The use of number of MI is still subjective, which is often encountered in the MCMC. And the MI may be more laborious when multidimensional latent trait were considered.

3. Is it of any advantages to transform the c parameter into logarithm form while running the estimation? And it put a log normal distribution on c parameter, looks like the 3PLM is inevitable to bad estimation when sample size is small, even the MI method was used.