ARC Laboratory Sharing

A Rasch Hierarchical Measurement Model

Author(s): Kimberly S. Maier

Journal of Educational and Behavioral Statistics, Vol. 26, No. 3 (Autumn, 2001)

This article introduces a model that combines an IRT with a hierarchical linear model referred to as Rasch hierarchical measurement model (Rasch HMM) to analyze the effect of multilevel covariates on a latent trait and a method of estimating model parameter values that does not rely on large-sample theory and normal approximations.

Rasch model connects observed responses and underlying unobserved latent trait.

HLM can study your interested effect covariates on the latent trait. Values for the parameters of the Rasch HMM were estimated using Markov Chain Monte Carlo (MCMC) techniques which are particular Bayesian data analysis methods. Gibbs sampling, as a specific MCMC technique, generates random variables from a distribution by sampling from the collection of full conditional distributions of the complete posterior distribution. Metropolis-Hasting method was used to draw samples from the latent trait and the item parameter’s conditional probability distributions. The common used priors of the estimated parameter were also presented in the paper.

Two simulated and one empirical datasets, were analyzed using the Rasch hierarchical measurement model. The first simulated data set is balanced and represents an ideal data situation. The second data set is an unbalanced sparse data set that would occur when a small and unequal number of measurements are made on a sample of students.

Simulation study:

The response strings for each of the two simulated data sets were generated in the following manner. First, values for the item difficulty parameters were generated from a standard normal distribution. Next, values of the latent trait parameters were generated using values of the level-2 intercept and level-1and level-2 error variances based on results from descriptive and IRT analyses of the data set constructed for the Maier (2000) study. The actual values used for the data simulation were 0.2835 for the level-1 error variance, 0.7099 for the level-2 error variance, and -0.0001 for the level-2 fixed intercept. Finally, the probability that a level-1 unit would answer an item correctly was calculated using the Rasch IRT model and the generated latent trait and item difficulty parameter values.

The third data set is a subset of data from the Sloan Study of Youth and Social Development and is similar to the data set used by Maier (2000). This data consists of responses of 313 adolescents collected while they were engaged in a mathematics classroom.

The statistics were calculated using CODA software

All three data sets were analyzed and the posterior distributions of the model parameters were produced using Gibbs sampling. The values of the Markov chains of each model parameter were used to generate the corresponding marginal distribution for each of the parameter estimates.

30,000 iterations of the algorithm were run. The first 1,000 iterations were considered to be the burn-in iterations and these corresponding deviates were discarded.

To illustrate how the Rasch HMM performs relative to a traditional two-step approach, the simulated balanced data set was reanalyzed. The result shows the two-step analysis approach grossly overestimates the level-1 random error variance and underestimates the level-2 random error

Variance while correctly estimating the level-2 fixed intercept.

Comments:

The result is not perfect for time-series plots for the level- I error variance for all the data sets show a lower rate of mixing, perhaps indicating that the Markov chain may not have converged.

There only two level in this research, if we add more level and expand the model to 2PL or more complicated model, I am worry about the time we need to estimate the data.

For me, I got a headache once I saw some many complex formulas.

More papers should be read to understand the MCMC method.