1. The authors assumed no significant correlation between theta and misconception. It means high-ability examinees can have misconceptions. I suppose that the high-ability examinees can rule out those misconceptions. That’s why they are of high-ability. Or what is the definition of high-ability in this study? That is, is the ‘ability’ they used equal to that we called before?
2. I associate the SICM with two-dimensional IRT model for “within-ability items”; that is, Pr(a1*theta1+ a2*theta2). But the right-answer category measures continuous theta, other wrong-answer categories measures discrete misconceptions. Note that the SCIM uses the compensation framework. I think the low ability and misconception cannot be easily distinguished. That may be why the reliability is low for theta even in large sample size and items. The authors suggest high quality of items are required (in Simulation Study Discussion), but they avoid to mention how to do for their model.
3. I am curious if the traditional model like PCM fit the responses generated from the SICM. What would happen? How would it affect the parameters related to theta (lamda)? If the estimates of thetas are similar to those of the SICM, it is advised to use the SICM.
4. The real data is a pretest. Did examinees never study the physics before the pretest?