The Representativeness Controversy

THE REPRESENTATIVENESS CONTROVERSY

The experiments summarized in Fig. 2.1 provided direct evidence for the representativeness heuristic and two concomitant biases: neglect of base-rates and conjunction errors. In the terminology introduced by Tversky and Kahneman (1983), the design of these experiments was “subtle”: adequate information was available for participants to avoid the error, but no effort was made to call their attention to that information. For example, participants in the Tom W. experiment had general knowledge of the relative base-rates of the various fields of specialization, but these base-rates were not explicitly mentioned in the problem. Similarly, both critical items in the Linda experiment were included in the list of outcomes, but they were separated by a filler so that respondents would not feel compelled to compare them. In the anthropomorphic language used here, System 2 was given a chance to correct the judgment, but was not prompted to do so.

In view of the confusing controversy that followed, it is perhaps unfortunate that the articles documenting base-rate neglect and conjunction errors did not stop with subtle tests. Each article also contained an experimental flourish – a demonstration in which the error occurred in spite of a manipulation that called participants’ attention to the critical variable. The engineer–lawyer problem (Kahneman & Tversky, 1973) included special instructions to ensure that respondents would notice the base-rates of the outcomes. The brief personality descriptions shown to respondents were reported to have been drawn from a set containing descriptions of 30 lawyers and 70 engineers (or vice versa), and respondents were asked, “What is the probability that this description belongs to one of the 30 lawyers in the sample of 100?” To the authors’ surprise, base rates were largely neglected in the responses, despite their salience in the instructions. Similarly, the authors were later shocked to discover that more than 80% of undergraduates committed a conjunction error even when asked point blank whether Linda was more likely to be “a bank teller” or “a bank teller who is active in the feminist movement” (Tversky & Kahneman, 1983). The novelty of these additional direct or “transparent” tests was the finding that respondents continued to show the biases associated with representativeness even in the presence of strong cues pointing to the normative response. The errors that people make in transparent judgment problems are analogous to observers’ failure to allow for ambient haze in estimating distances: A correct response is within reach, but not chosen, and the failure involves an unexpected weakness of the corrective operations of System 2.

Discussions of the heuristics and biases approach have focused almost exclusively on the direct conjunction fallacy and on the engineer–lawyer problems. These are also the only studies that have been extensively replicated with varying parameters. The amount of critical attention is remarkable, because the studies were not, in fact, essential to the authors’ central claim. In the terms of the present treatment, that claim was that intuitive prediction is an operation of System 1, which is susceptible both to base-rate neglect and conjunction fallacies. There was no intent to deny the possibility of System 2 interventions that would modify or override intuitive predictions. Thus, the articles in which these studies appeared would have been substantially the same, although far less provocative, if respondents had overcome base-rate neglect and conjunction errors in transparent tests.

To appreciate why the strong forms of base-rate neglect and of the conjunction fallacy sparked so much controversy, it is useful to distinguish two conceptions of human rationality (Kahneman, 2000b). Coherence rationality is the strict conception, which requires the agent’s entire system of beliefs and preferences to be internally consistent, and immune to effects of framing and context. For example, an individual’s probability P (“Linda is a bank teller”) should be the sum of the probabilities P (“Linda is a bank teller and is a feminist”), and P (“Linda is a bank teller and not a feminist”). A subtle test of coherence rationality could be conducted by asking individuals to assess these three probabilities on separate occasions under circumstances that minimize recall. Coherence can also be tested in a between-groups design. Assuming random assignment, the sum of the average probabilities assigned to the two component events should equal the average judged probability of “Linda is a bank teller.” If this prediction fails, then at least some individuals are incoherent. Demonstrations of incoherence present a significant challenge to important models of decision theory and economics, which attribute to agents a very strict form of rationality (Tversky & Kahneman, 1986). Failures of perfect coherence are less provocative to psychologists, who have a more realistic view of human capabilities.

A more lenient concept, reasoning rationality, only requires an ability to reason correctly about the information currently at hand, without demanding perfect consistency among beliefs that are not simultaneously evoked. The best known violation of reasoning rationality is the famous “four-card” problem (Wason, 1960). The failure of intelligent adults to reason their way through this problem is surprising because the problem is “easy,” in the sense of being easily understood once explained. What everyone learns, when first told that intelligent people fail to solve the four-card problem, is that one’s expectations about human reasoning abilities had not been adequately calibrated. There is, of course, no well-defined metric of reasoning rationality, but whatever metric one uses, the Wason problem calls for a downward adjustment. The surprising results of the Linda and engineer–lawyer problems led Tversky and Kahneman to a similar realization: The reasoning of their subjects was less proficient than they had anticipated. Many readers of the work shared this conclusion, but many others strongly resisted it.

The implicit challenge to reasoning rationality was met by numerous attempts to dismiss the findings of the engineer–lawyer and the Linda studies as artifacts of ambiguous language, confusing instructions, conversational norms, or inappropriate normative standards. Doubts have been raised about the proper interpretation of almost every word in the conjunction problem, including bank teller, probability, and even and (see, e.g., Dulany & Hilton, 1991; Hilton & Slugoski, 2001). These claims are not discussed in detail here. We suspect that most of them have some validity, and that they identified mechanisms that may have made the results in the engineer–lawyer and Linda studies exceptionally strong. However, we note a significant weakness shared by all these critical discussions: They provide no explanation of the essentially perfect consistency of the judgments observed in direct tests of the conjunction rule and in three other types of experiments: (1) subtle comparisons; (2) between-Ss comparisons; and, most importantly, (3) judgments of representativeness (see also Bar-Hillel and Neter, 1993, Chapter 3, this volume). Interpretations of the conjunction fallacy as an artifact implicitly dismiss the results of Fig. 2.1B as a coincidence (for an exception, see Ayton, 1998). The story of the engineer–lawyer problem is similar. Here again, multiple demonstrations in which base rate information was used (see Koehler, 1996, for a review) invite the inference that there is no general problem of base-rate neglect. Again, the data of prediction by representativeness in Fig. 2.1A (and related results reported by Kahneman & Tversky, 1973) were ignored.

The demonstrations that under some conditions people avoid the conjunction fallacy in direct tests, or use explicit base-rate information, led some scholars to the blanket conclusion that judgment biases are artificial and fragile, and that there is no need for judgment heuristics to explain them. This position was promoted most vigorously by Gigerenzer (1991). Kahneman and Tversky (1996) argued in response that the heuristics and biases position does not preclude the possibility of people performing flawlessly in particular variants of the Linda and of the lawyer–engineer problems. Because laypeople readily acknowledge the validity of the conjunction rule and the relevance of base-rate information, the fact that they sometimes obey these principles is neither a surprise nor an argument against the role of representativeness in routine intuitive prediction. However, the study of conditions under which errors are avoided can help us understand the capabilities and limitations of System 2.