Longitudinal studies often suffer from the loss of participants, which might influence the generalizability of research findings. Little and Rubin (2002) classified dropouts of a longitudinal study as MCAR, MAR, and MNAR, in the basis of independence of missing data on outcomes. Studies with dropouts have been found to yield biased estimates of parameters and model selections, resulting in inaccurate conclusions. Thus, this study tried to investigate the most appropriate information criterion for selecting a model fitting the data best. Manipulations in the present studies include types of mean and covariance structures, the repeated time points in a study, the sample size, the data distributions, and estimation methods.
A full and a reduced regression models were used to generate data in this study. A full model included a common intercept, a dummy group variable, a continuous covariate and a group and slope interaction; a reduced model consisted of a common intercept, a linear trend and a group covariate. Four covariance structures were selected to be paired with one of the regression model, including first-order autoregressive covariance, first-order autoregressive covariance with variance heterogeneity within subjects, Toeplitz covariance with variance heterogeneity within subjects, and unstructured covariance pattern.
Results indicated that no specific information criterion examined in the present study worked the best across all conditions. The performance of information criteria improved with increased sample size and the increased numbers of repeated assessments. As opposed to the findings of Gurka (2006), REML2 was found to function as well as ML did, but REML1 seemed to be worse than ML. None of the procedures performed well when data were moderately skewed.
Some limitations and suggestions for future studies are listed in the followings:
First, this study did not investigate the performance of widely used information criterion such as RMSEA. Although it stated clearly its focus on SAS, it is possible to obtain estimates from SAS for further calculations of other information criteria. It is recommended studying other information criterion in future studies. Second, it will be interesting to see how information criterion works when the model becomes non-linear. Third, explorations in the performance of a second-order structure model or in different covariance structures for participants in 2 or more treatment groups might be good issues for future studies. Fourth, the authors assumed that all participants responded to outcomes at exact time points in terms of ages, but sometimes it is not practical in longitudinal studies. If we free the time observations, will it change the conclusion? Lastly, when the data were moderately or seriously skewed, it might be more appropriate to use WLSM or other estimation methods, instead of ML or REML estimation. Those information is not available in the present study and needs further investigations.