Debiasing the Planning Fallacy

Debiasing the Planning Fallacy

Although the motivating effects of optimism may be worth the cost of misprediction in at least some situations (Armor & Taylor, 1998), there are many cases when accurate prediction is vital to success, such as when a new release of software is promised by a certain date. DeMarco (1982) defines an estimate as ”the most optimistic prediction that has a non-zero probability of coming true” (p. 14). In his field of specialty, optimistic predictions are the norm despite well-known rules of thumb such as the 90% rule (the last 10% of a project takes 90% of the resources). DeMarco estimates that 15% of software projects are never completed (justifying the term vaporware), and completed projects commonly cost two or three times as much as originally budgeted. For example, a General Accounting Office report cited in this account of U.S. military equipment procurement concluded that only 1% of major military purchases involving high technology were delivered on time and on budget. We investigated three approaches to reducing optimistic predictions: (1) reducing the dominance of the singular over the distributional approach by eliciting predictions for aggregate sets of events rather than single cases; (2) reducing the impact of ”planning for success” by encouraging the consideration of alternative, more pessimistic scenarios; and (3) linking the past to the future by requiring participants to explain how past experiences might be relevant to the current task.

The first debiasing approach, eliciting aggregate frequencies rather than single-case estimates, enjoys a substantial track record of success. Cumulative representations and responses reduce the conjunction fallacy in within-subject designs (Tversky & Kahneman, 1983) and the unpacking effect predicted by support theory (Tversky & Koehler, 1994), perhaps because the aggregate set serves as a reminder of the set-inclusion relationships that are the foundation of logic and probability theory (Griffin & Tversky, 1992; Kahneman & Tversky, 1982b). Thus, when predicting a set of future tasks, people may be attentive to their own set of past experiences or to the set of others’ experiences – that is, people may take a more outside perspective on the prediction process.

In our first two studies examining aggregate frequency judgments (Griffin & Buehler, 1999), participants listed 10 upcoming projects, provided best-guess completion times for each, estimated the probability that each project would be completed by the relevant best-guess date, and estimated how many of the 10 tasks would be completed by the relevant dates. In the third study, participants described a single project, estimated the probability that the project would be completed by the relevant best-guess date, and estimated how many of 10 similar tasks would be completed by relevant best-guess dates. In each study, the single-case probability and aggregate frequency estimates were counterbalanced for order. The results of all three studies failed to support the claim that frequency judgments are unbiased and follow normative principles (Cosmides & Tooby, 1996; Gigerenzer, 1991, 1994, 1998). Predictions, whether made for items or aggregates, were markedly over-optimistic and failed to discriminate between people who were more or less successful in task completion. The average item-specific confidence rating (73%) was almost identical to the expected aggregate frequency (7.2 out of 10), and the on-time completion rate was equally poor in both conditions (about 40% overall). Across all studies, only about 8% of the participants met or exceeded their predictions for the number of tasks they would complete.

Why was this manipulation such a failure? In accord with our analysis of the planning fallacy, the rich real-world prediction problems involved in these studies (e.g., “When will I buy a new bike? “ “When will I get my car repaired?”) may have masked the essentially statistical nature of such predictions. It takes a rather sophisticated view of the prediction process to see how a given task is like a “sample” from a population of tasks, and how the weight given to one’s plans must be moderated by considerations of the “predictive validity” of one’s plans in general (cf. Nisbett, Krantz, Jepson, & Kunda, 1983, Chapter 28, this volume). Instead, it seems that participants assessed the average “plan strength” for each problem, an inside estimation, and then calculated the average strength of all the plans in the set rather than starting with an outside view of the set of tasks.

The second debiasing manipulation, which involved the consideration of alternative scenarios (Newby-Clark et al., 2000), might look like a “sure thing” given the successful use of related techniques to reduce other judgmental biases such as overconfidence in knowledge and prediction (Koriat, Lichtenstein, & Fischhoff, 1980; Griffin et al., 1990; Hoch, 1985; Lord, Lepper, & Preston, 1984). Indeed, in many business and organizational contexts, where uncertain and uncontrollable events present serious difficulties for long-term planning, techniques involving multiple scenarios have become popular forecasting tools (for reviews, see Bunn, & Salo, 1993; Schoemaker, 1993). However, the failure of this technique in our studies bears testimony to the robustness of the planning fallacy and of the intuitive appeal of using best-case scenarios to guide time prediction. Participants’ best-guess predictions closely resembled predictions based on their most optimistic scenarios and were far more optimistic than those based on their pessimistic scenarios; furthermore, participants generally rated their optimistic scenarios as much more plausible than pessimistic ones, presumably justifying their neglect of the pessimistic scenarios. Even when participants were instructed to create pessimistic scenarios that were highly plausible, they continued to neglect these scenarios despite the fact that they could have resulted in increased accuracy. These findings also highlight, once again, the costs of basing forecasts on the plausibility of an imagined scenario: In this research, the very scenarios that seemed most plausible (the optimistic ones) were also the least accurate.

Our third debiasing manipulation required participants to link their past experiences with their specific plans for an upcoming task. In this ”recall-relevance” manipulation, participants first indicated the date and time they would finish a computer assignment if they finished it as far before its deadline as they typically completed assignments (Buehler et al., 1994, Study 4). Second, they described a plausible scenario – based on their past experiences – that would result in their completing the computer assignment at their typical time. This procedure should prevent participants from either ignoring past experiences with similar tasks or discounting the relevance of those experiences. After writing the hypothetical scenario, participants made predictions for the computer assignment. For this task, at least, the recall-relevance manipulation successfully eliminated the usual optimistic bias. Although this effect may have been an artifact of the relatively heavy-handed manipulation, it is worth remembering that the similarly heavy-handed demand for participants to construct a pessimistic scenario (Newby-Clark et al., 2000) utterly failed to reduce optimistic predictions. Although the mechanism is not entirely clear, the recall-relevance manipulation was the most effective of these interventions in the reported studies and deserves a trial in applied fields.