How much power and speed is measured in this test?
Ivailo Partchev
K. U. Leuven
Paul De Boeck
Power and speed are an old issue in psychological assessment. The author holds the opinion that the two are independent latent variable. The difficult for measure them is that they are bonded and test situation dependent, it is hard to distinguish them from the test for confounding factors in the items. Accuracy and response time were used for measuring power and speed.
The paper tries to use posterior time limits to avoid the confounding problem. To be specific, starting with the empirically recorded response times within a sufficiently large limit for all or almost all respondents, one can consider various time cut-offs. Only responses preceding the cut-off are counted as successes (if correct), and slower responses are considered either as failures or as missing.
The data is from the calibration studies for a computerized adaptive test developed by Hornke. Three binary data sets are produced：a time data set, a time accuracy data set, and an accuracy data set.
The results show that a reduction of the posterior time limit had no negative effect on the reliability of the latent trait measures (of whatever kind). And giving fast correct responses (accuracy data) is not more difficult than giving correct responses within a lenient time interval.
Comment1
If only use binary data, it seems that the data of response time is wasted. Will the response time be used as some validation criteria for speed？
Can the difference among the three data sets really reflect subjects’ power and speed？