A lot of studies have indicated the impact about choosing appropriate number of categories on reliability. This study concerns about the consistent psychological distance between categories. Wakita (2004) described a method of estimating the distance between categories based on the estimates of threshold parameters under IRT model. The new scale values are used to compare the value of threshold parameters. Then clarify how the number of options is enough in testing and how it affects the psychological distance.
The results in Figure 2 to Figure 4 shows that the converted item values seems inconsistent to psychological distance when the 7-category scale was used. The same results can be seen in Table 6.
Comments:
1. If the distribution of people is standard normal distribution as well, the standard normal distribution of category may be appropriate. However, the justification of this assumption may be still unclear in practice.
2. The quality of the scale item values are likely subjected to the standard error of the estimate of threshold parameters. The assumption of standard normal distribution of scaled value can be relaxed to let the variance being estimated. On the other hand, under the framework of MCMC, all the parameter estimation can run simultaneously.
3. I have no idea about the difference between conventional and converted item values. Does the absolute magnitude of difference make any sense in practice? They all are quite small numerically, although there is a larger difference when using 7-category scale than others.
4. It is possible to incorporate random threshold parameters (i.e., random error) to account for the inconsistent psychological distance.