Speed Matters, But Not How You Think: IQ & Latent Factors in Reaction Time

One of the more surprising findings to emerge from the intelligence literature is that an individual's ability to think in highly complex and abstract forms is related to speed in tasks as simple as "press the lighted button." Simple reaction time tasks like this have amazing predictive power for performance on much more elaborate tasks, leading some theorists to propose that such reaction time (RT) measures grossly index the integrity or speed of processing in a way that benefits all tasks.

Interestingly, the average speed on simple RT tasks is often not as predictive as other aspects of the RT distribution. In their new JEP:G paper on the topic, Schmiedek et al. describe the "worst performance rule" in which an individual's slowest reaction times are predictive of their short-term memory capacity, general intelligence, and processing speed (how quickly they can perform simple judgments).

Why might this be? Extraordinarily slow RTs may be a simple reflection of lapses in attention or goal maintenance. Thus the relative number of those lapses (or the time taken to recover from them) may be driving the relationship with complex task performance (where sustained attention and goal maintenance are clearly important).

A more sophisticated technique for estimating these lapses of attention or maintenance is the ex-Gaussian model, which Schmiedek et al also review in detail. This involves fitting a gaussian "bell-shaped curve" to reaction time data, but adding to it an exponential function that approximates an individual's slowest set of response times. This method has proven useful for modeling a large variety of RT distributions, which are almost always positively skewed - that is, the distribution is not perfectly bell-shaped around a mean, but instead has an extra helping of extremely long reaction times which tend to lengthen the bell-shape in that direction.

Schmiedek et al review theoretical interpretations of this exponential component, which have varied from interpretations of "central decision processes" to "goal neglect" to "attentional lapses." A very different approach to the skewedness of RTs comes from diffusion models, which (to simplify) posit that RT reflects two central parameters: the response criterion of the subject (how much evidence must be accumulated before the subject will provide a response) and the drift rate (the rate at which evidence accumulates as a function of its quality).

The effect of this is that those with slower reaction times in the gaussian component - disregarding the exponential component, for the moment - may actually perform better on many tasks, since in the diffusion model that reflects a more conservative response criterion (i.e., more evidence is required to make a decision). However, those with slower reaction times in the exponential component - disregarding or controlling for the gaussian component - may actually perform worse on more complex tasks, regardless of whether this reflects a slower accumulation of quality evidence (as suggested by diffusion models) or increased lapses in attention.

To test the counterintuitive prediction that different parts of the RT distribution should predict performance in opposite directions, Schmiedek et al analyzed data from 135 subjects who completed three types of tasks:

Choice Reaction Time: Eight different visually-presented 2-choice reaction time tasks (e.g., decide whether a word has one or two syllables, or whether a number is greater than or less than 500, etc);

Working Memory Capacity: Six measures of short-term or working memory capacity (including reading span, operation span, etc); and

Intelligence tests: Nine measures of reasoning and processing speed (from a standard IQ test).

Reaction times from each of these tasks were subjected to an ex-gaussian analysis, and were also fitted to the diffusion model to estimate response criterion and drift rate. Then, Schmiedek et al extracted consistent variation in each measurement across each type of task through the use of factor analysis. (Here the authors allowed the three error terms for each task to be correlated, and also allowed several residuals in the ex-gaussian parameter estimates for several choice-RT tasks to be correlated. The former suggests that some consistent source of variance is unaccounted for here - more on this issue later - and the latter suggests merely that the similarity of some of the choice-reaction time tasks may have been more than ideal for factor analysis).

The ex-gaussian results showed that only the latent exponential variable related to the working memory factor and to the reasoning factor, and that only the standard deviation of the latent Gaussian component related to the processing speed factor. Similarly, the diffusion model results showed that drift rate was the best predictor of the working memory factor, the reasoning factor, and the processing speed factor. As expected, the mean value of reaction time's gaussian component predicted accuracy on those tasks.

Contrary to the intuitive hypothesis that those with lower working memory should be slower on such simple reaction time tasks, these results demonstrate that this relationship is specific to the exponential component of the distribution, specificity that would not be expected from a "general slowing" perspective.

The authors go on to present multiple simulations of the data which indicate that the relationship between the exponential component and working memory need not rely on concepts like goal neglect or lapses in attention, since such a relationship is predicted by the diffusion model which doesn't invoke additional explanatory constructs like "goal neglect" or "attentional lapses" (the Razor strikes again). Instead, the relationship can be accounted for merely by the drift rate parameter.

But a question arises: what does the drift rate parameter itself reflect? Although it is motivated by theories of higher-order cognition, it's predictive capacity is more directly related to its mathematical instantiation, and the higher-order process it corresponds with is a empirical issue that is not directly addressed by the simulations here.

Schmiedek et al claim their results support the hypothesis that working memory is involved in "establishing and maintaining ad-hoc bindings between component representations" and that this is manifest in the drift rate through two contributing sources of variance: neural noise, and "robustness of temporary bindings." So, in the end, we see that this interpretation of the diffusion model does in fact invoke additional explanatory entities to explain "drift rate", and it's not clear to me whether neural noise and binding are better than goal neglect and attentional lapses. Binding is rarely studied in factor analyses, and neural noise may actually benefit the neural encoding of information due to a phenomenon known as stochastic resonance, demonstrated in some interesting information theoretic analyses of single cell data.

Nonetheless, the general take-home message is clear: simple reaction time is not so simple after all, and theoretically-informed statistical decomposition of reaction time distributions demonstrate that different aspects of this distribution have different relationships with higher-level cognitive constructs like intelligence, processing speed, and working memory capacity.