There are two key characteristics of repeated measures, missing data and observations within the same subject may be correlated. Many selection criteria were proposed in different software programs to select the most “true” model for data. This article examined the performance of selection criteria available in the major statistical packages in a situation with missing data and manipulated five variables: 8 types of mean and covariance structures, 2 number of repeated measurements per experimental unit, 3 total sample size, 2 population distribution shape, and 2 estimation methods resulted in total of 192 conditions. Five criteria were used in evaluated their performances in these conditions, and they are Akaike’s Information Criteria (AIC), Corrected Akaike’s Information Criteria (AICc), Bayesian Information Criteria (BIC) ,Consistent Akaike’s Information Criteria (CAIC), and Hannan-Quinn’s Information Criterion (HQIC). The first two criteria are belonging to efficient criteria and others are belonging to consistent criteria.
The results show that, first, the error rate for all criteria decreased as the sample size increased. Second, with small samples the standard selection criteria may be highly inefficient, particularly if some data are missing and/or the form of the matrix plays an important role in the estimation. Third, it should be noted that information criteria do not automatically select the best model from all possible candidate models. Finally, the results are of course limited to the conditions examined in our study.
Comments & Questions
1. This paper provides the detailed data generation and simulation procedure, and it also provide a briefly and clearly introductions of the criteria who were used in this study. Thus, it is very helpful to replicate the simulation study based on this paper.