57 Optimal Test Design with rule-based item generation (Present by Wayne on 18Mar2013))

cw's comments

cw's comments

by LIU CHEN WEI -
Number of replies: 1

1. At first, items belonging to which family requires classification by statistical method or expert’s judgments. It looks like a ‘classification’ problem except using model checking method mentioned in this paper.

2. P.142 it says certain combinations of radicals and incidentals may result in invalid item. Even using constraints to avoid the occurrence, how do we know its in validness? From practical or statistical view?

3. Are the beta parameters in Equation 4 estimated?

4. How many items were used in simulation one in page 148?

5. The mu(b) = -0.258 in page 148 looks strange. Do not know why it is so.

6. Compute the family information using MC method is straightforward. Is it possible using numerical integral? Because only three dimensions were used here.

7. It is better to know the practical variance of item parameters, then using them in simulation studies.

8. The Equation 6 requires to know the prior distribution of item parameters in advance. It is hard to know actually. If the distribution form is violated, what will happen to results?

9. Larger within variance in a family will lead to lower test information. What is the criterion to justify what is good or bad results?

In reply to LIU CHEN WEI

Re: cw's comments

by CHEN Chia Wen -

1. Author suggests us using expert's judgments. As xuelan's mention, actually the cognitive processes is impossible to be known.

2. really a problem

3. Is it a latent regression as the design matrix known?

4. yes, author didn't mention this in this artical even if in her dessertation in 2012.

5. see wolfe 1981, I think it is the point let the probability in 3P normal ogive equal to 0.5

6. I don't know now. it need to spend time to calculate.

8. yes, prior should be correct.

9. Do you think whether the lower information can be claimed a bad result under the information are correct calculated as family information?