Heuristics and Biases: Critiques and Controversies

CRITIQUES AND CONTROVERSIES

The profile of any intellectual idea is also raised by the controversies it inspires, and the heuristics and biases tradition has inspired many. People, particularly academics, do not accept new ideas and approaches easily, nor should they. As Galbraith noted, “Faced with the choice between changing one’s mind and proving that there is no need to do so, almost everyone gets busy on the proof.” So it has been with the reaction to the heuristics and biases program – many minds have been busy defending the rationality of everyday judgment and proving that the core ideas of the heuristics and biases program are misguided. Here are the central ideas of some of those proofs.

The “We Cannot Be That Dumb” Critique. The most common critique of the research on heuristics and biases is that it offers an overly pessimistic assessment of the average person’s ability to make sound and effective judgments. People by and large manage their lives satisfactorily, something they would be unlikely to accomplish, the argument goes, if their judgments were so prone to bias. Indeed, working collectively, humans have split the atom, recombined DNA, and traveled to the moon. Critics see the heuristics and biases program as denigrating “human decision makers as systematically flawed bumblers” (Ortmann & Hertwig, 2000) because “actual human reasoning has been described as ‘biased,’ ‘fallacious,’ or ‘indefensible’” (Gigerenzer, 1991a, p. 259). As an outraged team of psychologists queried, “Are heuristics-and-biases experiments cases of cognitive misers’ underachieving, or of their receiving a Bayesian hazing by statistical sophisticates?” (Barone, Maddux, & Snyder 1997, p. 143).

This critique owes much of its pervasiveness and appeal to the fanfare that the negative message of the heuristics and biases program has generated at the expense of its positive counterpart. There is, of course, some inevitability to this: Negative information typically dominates the positive. Just as there is a “bad news bias” in media reporting (“if it bleeds, it leads”), it is hardly surprising that the negative message of the heuristics and biases program would capture more attention, inspire more like-minded research, and serve as the focal point of disagreement and controversy. Nonetheless, the common belief that examples of human error are disproportionately cited in the scholarly literature turns out to be an oversimplification; the prominence of such demonstrations is accounted for by the prominence of the journals in which they are found (Robins & Craik, 1993).

There is, however, one version of this critique to which researchers in the heuristics and biases tradition must plead “no contest” or even “guilty.” This is the criticism that studies in this tradition have paid scant attention to assessing the overall ecological validity of heuristic processes. Ecological validity (Brunswik, 1955) corresponds to the correlation of the actual outcome in the world with the cue available to the perceiver across a universe of situations. Thus, assessing the ecological validity of the representativeness heuristic would involve identifying a universe of relevant objects (e.g., every scholar in the engineering and law faculties at a given university), and then correlating the outcome value for each object (e.g., membership in either faculty) with the value of the cue variable for each object (e.g., relative similarity to the prototype of each faculty). This correlation, then, would provide a measure for the given universe of how well the representativeness cue performed. This Herculean task has not attracted researchers in the heuristics and biases tradition; the focus has been on identifying the cues that people use, not on evaluating the overall value of those cues. Nevertheless, researchers in this tradition clearly share a set of assumptions: the ecological validities are probably high, the heuristics are generally useful, but common and profoundly important exceptions are to be found. (Note how this summary could be applied to the “fast and frugal” decision heuristics discussed in Chapter 31 by Gigerenzer et al., despite the apparent opposition between the ecological rationality movement and the heuristics and biases perspective.)

Thus, although there is doubtless some scorekeeping with respect to instances of sound and unsound judgment, it is not of the “box score” sort in which a tally is kept of the number of instances in which people exhibit biased and unbiased judgments. Such a tally is beside the point. A meaningful overall characterization of the quality of human judgment is neither possible nor sought after. To the extent that any such characterization is possible, it would be hard to resist the conclusion that the glass is both half full and half empty. People make a great many judgments with surpassing skill and accuracy, but evidence of dubious belief, questionable judgment, and faulty reasoning is not hard to find (Dawes, 1988; Gilovich, 1991; Schick & Vaughn, 1999; Stanovich, 1986).

Note that there is more than a little irony in the strong form of this critique, one that advances the rather Panglossian notion that people’s judgments are hardly ever biased (see Stanovich & West, 2000; Chapter 24, for a consideration of this view). The same scientists who advance this claim use a variety of methodological safeguards such as double-blind experimental designs to make sure their own observations are not contaminated by bias. Are the observations of scientists so much more prone to bias than the individuals they study?

Advocates of the “people are not that dumb” critique have found their voice among evolutionary psychologists for whom it is axiomatic that people perform all tasks critical for survival and reproduction extremely well. According to this school, ancestors who could not reliably make judgments important to survival did not survive long, and therefore the biological basis of their judgmental tendencies have been driven from the gene pool. There is, of course, considerable merit to this perspective. Only a creationist would maintain that our mental faculties were sculpted by something other than evolution. It is also the case that some judgments strike us as hard and others easy, and it is a good bet that the ones that strike us as hard were not subject to the same intense evolutionary pressures as those that strike us as easy. The problems our ancestors absolutely had to solve offer little challenge to us now because of the mental mechanisms we inherited to solve them.

But this logic hardly implies that there is no room for systematic error in judgment. Evolutionary pressures acting on the bulk of human judgments are neither sufficiently direct nor intense to sculpt the kind of mental machinery that would guarantee error-free or bias-free judgment. As Simon pointed out long ago (1956, 1957), evolutionary pressures only lead to local (“better than”), not global (“best possible”) optimization. Evolutionary pressures lead to adaptations that are as good or better than a rival’s; they do not lead to adaptations that are optimal. If they did, warblers would not rear cuckoo chicks (which they do even though the cuckoo chick is much bigger than the adult warbler), lions would not stalk upwind of their prey (which they do despite greater success stalking downwind), and people would not probability match in so many different domains (which they do despite paying a high price in foregone gains).

It is ironic that the heuristics and biases approach would be criticized as inconsistent with the dictates of evolution because it is an evolutionary account (see Chapter 2 in particular). It is an evolutionary account that recognizes the constraints imposed by an organism’s evolutionary history, constraints that yield noteworthy imperfections in function. As Gould (1997, p. 52) argued, “even the strictest operation of pure Darwinism builds organisms full of nonadaptive parts and behaviors. . . . All organisms evolve as complex and interconnected wholes, not as loose alliances of separate parts, each independently optimized by natural selection.” The heuristics and biases approach takes the notion of such historical constraints seriously and examines the imperfections that both reveal that history and illuminate current mechanisms.

Kahneman and Tversky’s frequent analogies between perceptual and cognitive processes highlight this historical emphasis and reflect an important recognition that cognition evolved after (and out of) perception. Organisms must perceive and act before – or more pressingly than – they need to think, and this doubtless has implications for the quality with which these functions are carried out. Compare, for example, the quality of your motor memory with the quality of your semantic memory. Compare how easy it is to remember how to ski decades after last doing so with how easy it is to recall the trigonometric functions you learned in high school, the foreign language you learned as a teenager, or even all of your childhood romances.

It is clear, then, that there is no deep-rooted conflict between an evolutionary perspective on human cognition and the heuristics and biases approach (Samuels, Stich, & Bishop, in press). Both are concerned with understanding the psychological mechanisms people employ to solve a variety of important real-life problems. Both acknowledge that many cognitive problems essential to survival are typically solved with efficiency and precision. And both can accept the existence of pockets of (particularly informative) bias and error in human judgment.

Indeed, even one of the more popular metaphors of the evolutionary approach to reasoning – that of the mind as a Swiss Army knife – is entirely consistent with the heuristics and biases approach. Although psychologists and neuroscientists have no handle on just how modular the mind might be (Fodor, 2000), it is certainly not unreasonable to suppose that many higher-order cognitive functions are indeed performed by discrete modules. There might be, for example, a module that computes similarity between entities, another that performs basic counting and frequency functions, another that handles causal relations, and so on. Such a mind – one that used different “tools” to perform its various tasks – would produce a pattern of judgments that corresponds perfectly to that documented in the heuristics and biases tradition. At some times and in some contexts, tasks are performed by just the right module and sound judgments are made. At other times and in other contexts, however, specific tasks are coopted by the wrong module and systematically biased judgments are the result. On still other occasions, of course, the mind might not have the right module to handle the problem (no Swiss Army knife does everything an outdoorsman needs done) and so the task is assigned to a “next best” module, and imperfections in judgment should once again be the result. A modular mind should also produce a pattern of judgments whereby a problem described or structured in one way yields one type of response, whereas the same problem described or structured another way yields a vastly different response – exactly the pattern of results reported countless times in the heuristics and biases literature.

The “It’s All Parlor Games” Critique. Another common critique of the heuristics and biases tradition has been to dismiss the reported findings as mere laboratory curiosities – as demonstrations that people cannot readily solve tricky “word problems.” The implication is that judgment outside the laboratory is likely to look far superior to that exhibited within.

This critique overlooks that it was the existence of biased judgments in the real world that motivated the heuristics and biases research program. Recall that an important impetus for this research was the work by Paul Meehl on the problems inherent in expert clinical judgment. Recall also that it was the observation of faulty reasoning among students trying to learn statistics (e.g., the gambler’s fallacy, the regression fallacy, insensitivity to sample size) that gave the research its initial shape. This critique also flies in the face of the influence that the heuristics and biases research program has had across a wide range of applied disciplines, something it could not do if it dealt only with contrived, artificial problems. As we have noted, the heuristics and biases program has influenced scholarship and curricula in political science, medicine, law, and management.

One particularly persistent form of this critique is the claim that the biases revealed in this research are merely the product of fiendish (or clueless) experimenters who ask misleading questions. Participants are not responding incorrectly, in other words; they are giving the right answer to a different question than the one the experimenter believes he or she is asking.

There is doubtless some merit to this claim, at least as applied to some individual experiments that purport to demonstrate a given bias or shortcoming of human judgment. There is a complex psychology – a subtle set of tacit assumptions and implicit demands – that accompanies participation in a psychology experiment. Even investigators attuned to this psychology can sometimes fail to anticipate correctly how a stimulus is likely to be construed or a question interpreted by a given participant. It is no small task for experimenters to “get it right,” which is why psychological research requires so much pretesting.

But just as it is clear that some individual experiments are open to this critique, it is equally clear that the main biases uncovered in this research tradition (e.g., availability biases in likelihood estimates, insensitivity to sample size and prior probability, the conjunction fallacy, anchoring, packing and unpacking effects) are not. These have all been demonstrated in countless contexts and with varied paradigms and dependent measures, and with domain experts as well as student volunteers. Although an isolated demonstration of some of these biases may be open to this critique, the overall body of evidence in support of them is not. For example, in one of the original demonstrations of the conjunction fallacy using the famous “Linda problem,” it is entirely possible that participants interpreted the option “is a bank teller” to mean “is a bank teller who is not active in the feminist movement” given that one of the other options was “is a bank teller who is active in the feminist movement.” Participants who construed the former this way can hardly be faulted for ranking it lower in likelihood than the latter. But this alternative interpretation simply cannot handle the observation that when participants in a between-subjects design rated (rather than ranked) the likelihood of only one of these options, those evaluating Linda the feminist bank employee offered higher likelihood ratings than those evaluating Linda the bank employee. In such a between-subjects format, of course, there is no artifactual basis for participants to conclude that “bank teller” is to be interpreted as “nonfeminist bank teller.” To the extent that any participants did so, it was because their mental model of bank teller crowded out the possibility of feminism, not because they could infer that the experimenter intended “bank teller” to mean “nonfeminist bank teller.” (Chapters 1 and 2 contain more detailed discussions of alternative interpretations of the conjunction fallacy.)

This example raises the important point that certain misconstruals on the part of participants are not artifacts, they are the phenomena of interest. There is a long tradition of research in social psychology illustrating that people actively construe the meaning of a given task or stimulus (Griffin & Ross, 1991) and that their own chronically accessible categories, habits, and experiences powerfully influence their construals (Higgins, King, & Mavin, 1982; Higgins, Rholes, & Jones, 1977; Srull & Wyer, 1979, 1980). Consider Kahneman and Tversky’s well-known engineer/lawyer problem. When asked whether a given description is likely to belong to an engineer or lawyer, one cannot fail to compute the similarity between the description and each professional stereotype. Both the immediacy and relevance of this assessment of similarity, then, make it highly likely that one’s very definition of what the task is about will be hijacked. A question about likelihood is construed as a question about “fit.” And there is nothing artifactual about the switch (on this point, see Kahneman & Frederick, Chapter 2; Bar-Hillel and Neter, Chapter 3, this volume). It happens both inside and outside the laboratory, and, just as one would expect if people were active interpreters of the tasks that confront them, various changes in the presentation of stimuli or the description of the task influence what participants interpret their task to be (e.g., Koehler, 1996; Macchi, 1995).

The “It’s Not an Error” Critique. Another common accusation against the heuristics and biases tradition is that researchers hold experimental participants to an inappropriately high or even misguided standard of rationality. Jonathan Cohen, for example, contends that rationality does not exist apart from human intuition (Cohen, 1981) – in fact, standards of rationality are the “distillation of our best intuitions about rationality” (Papineau, 2000, p. 173). Thus, how can the source of our standards of rationality prove to be irrational? Perhaps people – especially people participating in unfamiliar or otherwise misleading experimental games – make performance mistakes that mask their underlying rational competence, but by definition, human intuition must be rational.

This critique usefully points out two aspects of the “rationality problem.” First, it has a distinct “Alice in Wonderland” flavor: People can and do define rationality in many contradictory ways (see Evans and Over, 1996, for an attempt to deal with the definitional problem). Second, it brings to the fore the crucial role of axioms in justifying a normative theory. Probability theory is built from a few fundamental rules or axioms that reflect people’s considered opinions in the abstract. These axioms include the claim that the probability of an event A and its complement not-A sum to 1. This is an example of a coherence axiom that constrains the relation between the probabilities of events. Axioms have force only if people agree with them – it is possible to opt out and agree not to be bound, but most people find such rules to be compelling. In fact, it is the tension between the general agreement with the abstract rules of probability and the violation of those rules in richer contexts that give the heuristics and biases demonstrations their power (a point explored more deeply in Chapters 1, 2, and 22).

A number of prominent statisticians and philosophers have opted out from any version of probability theory that deals with unique or “one-shot” events. Supporters of the “objective frequency” or “relative frequency” approach (e.g., von Mises, 1928) restrict the domain of probability theory to events that can be repeated in an infinite series. According to this perspective, probability is defined as the relative frequency of an event in an infinite series. Other types of events – such as predictions, beliefs, or statements about a single case – cannot be evaluated by the rules of probability theory. There is no denying the appeal of dividing the domain of uncertainty into the “objective” (like the spin of a roulette wheel or the toss of a coin) and “subjective” (like a prediction of tomorrow’s weather). However, as Keynes (1921) argued, a strict frequentist view entails that beliefs about unique events such as the coming of war, the end of a recession, and the outcome of a medical operation cannot be evaluated. And even those who take the frequentist stance in their professional lives act like subjectivists in their day-to-day affairs. An honest frequentist must concede that meaningful probabilistic statements can be made about unique events, such as the Yankees being more likely to win the World Series this year than, say, the Kansas City Royals or Montreal Expos, or that either author of this chapter is likely to lose a prize fight with the reigning champion in his weight class.

Such consideration notwithstanding, at some point one is thrown back to the level of axioms: Is one willing, for example, to subscribe to the rule that a set of predictions made with 80% probability should come true 80% of the time? For those willing to opt in to such a “correspondence axiom,” the demonstrations found within this book should be particularly relevant. However, even those readers who by personal taste or ideological alliance reject the correspondence axiom might be swayed by the “expected value” argument as characterized by the statistician De Finetti. “The formal rules normally used in probability calculations are also valid, as conditions of consistency for subjective probabilities. You must obey them, not because of any logical, empirical or metaphysical meaning of probability, but simply to avoid throwing money away”(De Finetti, 1970, p. 137). Finally, those frequentists who are unwilling to have their own beliefs evaluated by the coherence or correspondence axioms may still be curious to find out how well the classical rational actor model – incorporating the axioms of subjective probability – stands up to empirical test.

The “Frequencies, Good; Probabilities, Bad” Critique. Given the controversy surrounding the normative status of frequencies and subjective probabilities, it is not surprising that those who favor an evolutionary defense of rationality (“ecological rationality”) should throw in their lot with the frequentists. Evolutionary psychologists (e.g., Pinker, 1997) maintain that success in our ancestral environment required only a talent for working with frequencies, not probabilities. This argument, precisely because it cannot be tested empirically, remains a matter of faith and ideology. However, the frequentist argument for evolutionary rationality contains a component that can be tested empirically: The evidence for heuristics and biases, it is claimed, “disappears” when stimuli are presented and questions are asked in terms of frequencies (Gigerenzer, 1991b; 1994).

This was a bold argument when first introduced and it is even bolder to maintain now (e.g., Gigerenzer, 1998; Cosmides & Tooby, 1996) when a score of studies have indicated that it simply does not hold up empirically. In fact, presenting frequencies rather than probabilities sometimes makes judgment distinctly worse (e.g., Griffin & Buehler, 1999; Treadwell & Nelson, 1996), sometimes makes judgments distinctly better (e.g., Tversky & Kahneman, 1983; Koehler, Brenner, & Tversky, 1997) and quite often leaves the quality of judgment largely unchanged (Brenner, Koehler, Liberman, & Tversky, 1996; Griffin & Buehler, 1999). Even more troublesome for the evolution/frequency argument, Kahneman and Tversky’s original explanation of the probability–frequency discrepancy (Kahneman & Tversky, 1982a; Tversky & Kahneman, 1983) provides a unified account of when frequency formats improve judgments and when they do not (e.g., Sloman, Slovak, & Over, 2000).

Critics claim that assessments of single-event probabilities are unnatural, and that only a frequency format is consistent with how the mind works (Cosmides & Tooby, 1996; Gigerenzer, 1991b, 1994; Pinker, 1997). Kahneman and Tversky argued, in contrast, that representing problems in terms of frequencies tends to evoke mental models that facilitate the detection of set inclusion relations and thus improves judgment – and this view has received considerable support from the studies of Sloman and others (e.g., Evans, Handley, Perham, Over, & Thompson, 2000; Girotto & Gonzalez, 2001; Sloman & Over, in press; Sloman et al., 2000).

Note that even some scholars who are sympathetic to the heuristics and biases tradition have concluded that its lessons are limited to “problems with probabilities.” “While we have had the intellectual resources to pursue truth for at least 100,000 years, and quite possibly a lot longer, the notion of probability has only been around since 1654. . . . It is no accident that most of the ‘irrationality’ experiments trade in probabilities” (Papineau, 2000, p. 182). Thus, it is important to point out that although frequentistic formats – for whatever reason – sometimes induce more effective processes of judgment, it is simply not the case that the biases uncovered by Kahneman, Tversky, and others “disappear” if people are allowed to think in terms of frequencies rather than probabilities. Numerous studies in the heuristics and biases tradition make this clear, any one of which is sufficient to make the point. For example, in one of the earliest of Kahneman and Tversky’s experiments, participants were asked to estimate either the number of possible committees of 2 people from a group of 10, or the number of possible committees of 8 from a group of 10. Here, as in many other studies, the participants were given no information in a probabilistic format nor was a probabilistic response required (or even possible). Nevertheless, even though the actual number of possible committees of 2 and 8 are the same, those estimating the number of 2-person committees gave estimates that were an average two and a half times higher than those estimating the number of 8-person committees (Tversky & Kahneman, 1973). There is clearly more to biased judgment than an inability to handle probabilities.