The study focused on the issue of overextracting classes in mixture modeling and the factors contributing to the spurious classes.
1. Exploratory application of an MRM to an empirical example
The exploratory MRM analysis was first conducted to the data with all items, and it was determined there are three latent classes because the BIC of three classes are the smallest in most of the samples. However, when the test was first calibrated with the 3PL model and the item which are found to be underfit were excluded, it was found that the better solution become to be two classes because now the BIC of the two class is the smallest. Therefore, it was concluded that the underfit items appear to be involved in creating extra classes in MRMs.
Q1: The BIC is, actually, very close between two class and three class in the analysis with all items and with the underfit items removed (Table 1). For example, there are 99790 and 99795 for the two class solution and three-class solution in the analysis after the underfit items removed. Is it convincing that the better model is the two-classes solution?
Q2: The fit statistic Q-index is rarely seen in the paper. How about its reliability?
2. Simulation Study 1: Using a mixture Rasch model on one-class 2PL data
When the data was generated with the one-class 2PL model, which is obviously violates the Rasch model, it was found that the MRM produced two or more classes, especially in the conditions with more items and the large sample size.
Q3: The mixture model was found to perform better with large sample size (e.g., Maij-de Meij, Kelderman & van der Flier, 2010). But it was found that it is more likely to have spurious latent classes. How to interpret the controversial?
Q1: If the data is generated from one-class Rasch model, will the MRM definitely give the one-class solution? Since there was no unfit item which will cause the spurious latent class.
Q2: It was found an interesting observation about the locations of ICCs in the two class MRM (p. 320). But what’s the meaning for the interesting phenomena?
Q3: It was concluded that the one-class Rasch model estimated a 2PL ICC as well as a two-class MRM, and therefore, there is no need for two classes. But, the aggregated of two Rasch curves with fixed proportion and varied proportion is hard for me. Anyone could explain more?
3. Simulation 2: How many non-Rasch items would be needed to cause a spurious class?
It was found that only one non-Rasch items with high discrimination parameter for large sample is sufficient to cause a second class to be detected.
Q: The MRM is sensitive to the violation of Rasch model in large sample size, which is not desirable, however.
4. Simulation 3: Do two classes in a MRM always collapse into one class in 2PL?
Q: When using a MRM to fit a one-class 2PL data, I expected to see it will produce two-classes initially and then obtain one class after the unfit items were removed.
To me, it is the continue of the simulation 1. But how to find the unfit items. To calibrate the 2PL data with Rasch model? I’m not sure.