Performance Errors and Computational Limitations

PERFORMANCE ERRORS AND COMPUTATIONAL LIMITATIONS

We view the magnitude of the correlation between performance on a reasoning task and cognitive ability as an empirical clue about the importance of algorithmic limitations in creating discrepancies between descriptive and normative models. A strong correlation suggests important algorithmic-level limitations that might make the normative response not prescriptive for those of lower cognitive capacity (Panglossian theorists drawn to this alternative explanation of normative/descriptive gaps were termed Apologists by Stanovich, 1999). In contrast, the absence of a correlation between the normative response and cognitive ability suggests no computational limitation and thus no reason why the normative response should not be considered prescriptive (see Baron, 1985).

In a series of investigations (Sá, West, & Stanovich, 1999; Stanovich, 1999; Stanovich & West, 1997, 1998c, 1999) we have found moderate correlations between measures of cognitive ability and several tasks well-known in the heuristics and biases literature (e.g., informal argument evaluation tasks, belief bias in syllogistic reasoning, covariation detection, causal base-rate use, selection task performance). In addition, much smaller but still significant correlations were observed between cognitive ability and a host of other tasks in this literature (e.g., assessing the likelihood ratio, sunk cost effects, outcome bias, “if only” thinking, searching for unconfounded variables). Finally, there are some tasks in the heuristics and biases literature which lack any association at all with cognitive ability – the so-called false consensus effect in the opinion prediction paradigm (Krueger & Clement, 1994; Krueger & Zeiger, 1993) and the overconfidence effect in the knowledge calibration paradigm (e.g., Lichtenstein, Fischhoff, & Phillips, 1982).

These results indicate that although computational limitations appear implicated to some extent in many of the tasks, the normative responses for all of them were nonetheless arrived at by some university students who had modest cognitive abilities (e.g., below the mean in a university sample). For most of these tasks, only a small number of the students with the very lowest cognitive ability in these samples would have prescriptive models that deviate substantially from the normative model for computational reasons. Such findings might be taken to suggest that perhaps other factors might account for variation in the magnitude of the normative/descriptive gap – a prediction confirmed in our work on styles of epistemic regulation (Sá et al., 1999; Stanovich & West, 1997). Thus, the failure of performance errors and computational limitations to completely explain the normative/descriptive gap (on the former, see chapters 2 and 6 of Stanovich, 1999) implies that there may well be true individual differences in intentional-level psychologies – that is, true individual differences in rational thought.

All of the camps in the dispute about human rationality recognize that positing computational limitations as an explanation for differences between normative and descriptive models is a legitimate strategy. Meliorists and Panglossians alike agree on the importance of assessing such limitations. A more controversial alternative is that inappropriate normative models have been applied to certain tasks in the heuristics and biases literature.