Examining Rounding Rules in Angoff-Type Standard-Setting Methods
The purpose of the present study is to investigate how different rules adopted here in Angoff-type standard setting methods affect cut scores. The authors briefly introduced common modifications of the Angoff procedure. The first one is to extend to other rounding rule than just use a whole number of score points. The second one is to ask panelists to provide judgments across clusters of items than to each item. The third one is whether treat mean or median of the panelist judgments affect group cut score estimates. Therefore, the authors conducted different rounding rules and ways of providing Angoff standard setting judgment to examine how well aforementioned factors affect cut scores. A series of simulations were carried out, treating parameters calibrated by real data as true values and compared estimate and true value. Results illustrated that rounding to the nearest whole number could severally affect the cut scores for both individual panelists and a group of panelists. However, results didn’t show that the nearest whole number rounding rule would always produce larger bias.