53 Effects of multiple testing adjustment in DIF detection (Present by Jacob)

Xue-Lan's review

Xue-Lan's review

by QIU Xuelan -
Number of replies: 0

Three adjustments procedures (Bonferroni, Holm's procedure, and BH's procedure) were used into four DIF detection methods and compared their performance. I'm asking myself two questions before reading the article: What are the advantages and disadvantages of the adjustment procedures in DIF analysis? What if these procedures can perform well?

The most advantage of the procedures maybe, I think, is convenient and some (Holm and BH) can perfor well under some conditions. However, it decrease power sustantially, especially for Bonferroni, and when sample size are relative small.

The interesting finding of the Type I error rate of the DFIT and the Lord's chi-square test, may due to the simulation design that may not lead to inflated Type I error for these two methods. The percentage of DIF items are 15% among 20/40 items. And the DIF amount are 0.7/0.5/0.3. Following the ASA (averaged signed area), the largest 'averaged' DIF effect are actually small (0.7*6/20=0.21). That's the reason that I guess why the Type I error are not inflated before correction.

The reseachers should investigate the condition in which the ASA is large enough for the DFIT and Lord's chi-square test to verify their findings.