This study aimed to test the robustness of multilevel linear growth curve modeling. One current evidence available in the literature random effects were poorly estimated under many conditions, even under correct model specification. Repeated measurements of a participant across time may be modeled for two stochastic processes: autoregressive (AR) and moving average (MA). In addition, these two processes can be combined, resulting in an autoregressive moving average (ARMA) process. Their research questions are: (a) How well are fixed and random effect parameters and Type I error rates of the tests of fixed effects estimated when correct and incorrect error specifications are used? (b) How effectively do model fit indices work in identifying the correct covariance structure specification? and (c) How well does the unstructured error matrix perform when it represents a misspecification? Monte Carlo methods were used to generate data with SAS/IML and analyzed with the SAS PROC MIXED procedure. Four factors were systematically varied in generating the data, 1) the autocorrelations of the AR (1) process, 2) the correlations of the MA, 3) sample size, 4) series length resulting 16 cells, where each cell has a total of 10,000 data sets.
To evaluate the results, relative biases were evaluated for fixed effects Type I error rates Fit indexes AIC, BIC, AICC, Effect sizes. Implications of the study included warnings:
1) for applied researchers in whether they rely on testing variance components to decide whether explanatory variables should be added to a growth model.
2) For trying out different covariance matrixes and relying on fit indexes to indicate the correct model.
However, for researchers interested in making inferences about growth curve model fixed effects only, the negative results seem to have little impact. Further studies directions for researchers are that they should consider other possibilities. Second, the effects of covariance misspecification in combination with violations of other statistical assumptions, such as nonnormality, can be examined.