The purpose of the paper was to examine whether or not speed and power are different cognitive performance traits and thus, the authors proposed an approach based on posterior time limits to differentiate speed and power measured in a test. Accuracy and response data were recoded to three data files with binary categories: time, time-accuracy, and accuracy. The 2-PL model was conducted to estimate item difficulties and discriminations, correlations of the 3 kinds of data with various posterior time limits were compared to the anchors, a principal components analysis was used to examine if only two dimensions existed in the datasets. Finally, coefficient was calculated to reflect speed in the verbal analogies test.
The results of PCA suggested that power and speed could be considered as two different cognitive traits, but they were not completely unrelated. Time data were correlated with speed, but the relation decreased as the posterior time limit became lenient. Time-accuracy data moved closer to speed from power when the limit decreased. Accuracy data were located near power and were not influenced by the changes of posterior time limits. Reliability increased as the time limit became more stringent, suggesting that lucky guesses might be reduced when time limits reduced.
Overall, it is an interesting paper which mainly focuses on psychological properties of test takers. The authors made good use of accuracy and response time to differentiate power and speed in a test, and neither complex statistical methods nor complicated models were used to estimate test takers’ performance or test properties. It will be very interesting and meaningful if motivation, stress, anxiety or participants’ personality are taken into account with a study of power and speed and therefore test scores will be more capable of reflecting take takers’ latent abilities.
The followings are my questions:
1) On page 11, there seems to be a typo error in the variable name of accuracy data.
2) Cross-loadings were found in a principle components analysis. It might imply that there was no clear cut between power and speed, or that the use of posterior time limit is not good enough to differentiate power and speed in an intelligence test.
3) It is not very clear to me why the estimated linear and quadratic trends in item difficulty and discrimination could be used to estimate a function of posterior time limits for 3 types of data.