Checking Intuitive Predictions

“That start-up achieved an outstanding proof of concept, but we shouldn’t expect them to do as well in the future. They are still a long way from the market and there is a lot of room for regression.”

“Our intuitive prediction is very favorable, but it is probably too high. Let’s take into account the strength of our evidence and regress the prediction toward the mean.”

“The investment may be a good idea, even if the best guess is that it will fail. Let’s not say we really believe it is the next Google.”

“I read one review of that brand and it was excellent. Still, that could have been a fluke. Let’s consider only the brands that have a large number of reviews and pick the one that looks best.”

Part 3: Overconfidence

19. The Illusion of Understanding

Daniel Kahneman’s account draws on Nassim Nicholas Taleb’s discussion of the narrative fallacy.

The trader-philosopher-statistician Nassim Taleb could also be considered a psychologist. In The Black Swan, Taleb introduced the notion of a narrative fallacy to describe how flawed stories of the past shape our views of the world and our expectations for the future. Narrative fallacies can arise as we try to make sense of the world. The explanatory stories that people find compelling are simple; are concrete rather than abstract; assign a larger role to talent, stupidity, and intentions than to luck; and focus on a few striking events that happened rather than on the countless events that failed to happen. Any recent salient event is a candidate to become the kernel of a causal narrative. Taleb suggests that we humans constantly fool ourselves by constructing flimsy accounts of the past and believing they are true.

The narrative fallacy is fundamentally a problem of information reduction. Our minds have a limited ability to look at sequences of facts without weaving an explanation into them—forcing a logical link, an arrow of relationship, upon them. Explanations bind facts together, making them more easily remembered and helping them make more sense. This propensity goes wrong when it increases our impression of understanding without improving its accuracy. We like stories, we like to summarize, and we like to simplify, i.e., to reduce the dimension of matters. While this urge may stem from an ingrained biological need, it is so deep that even an information-processing robot would be prone to the same reductive process. Information wants to be reduced.

Good stories provide a simple and coherent account of people’s actions and intentions. You are always ready to interpret behavior as a manifestation of general propensities and personality traits—causes that you can readily match to effects. The halo effect contributes to coherence, because it inclines us to match our view of all the qualities of a person to our judgment of one attribute that is particularly significant. If we think a baseball pitcher is handsome and athletic, for example, we are likely to rate him better at throwing the ball, too. Halos can also be negative: if we think a player is ugly, we will probably underrate his athletic ability. The halo effect helps keep explanatory narratives simple and coherent by exaggerating the consistency of evaluations: good people do only good things and bad people are all bad. The statement “Hitler loved dogs and little children” is shocking no matter how many times you hear it, because any trace of kindness in someone so evil violates the expectations set up by the halo effect. Inconsistencies reduce the ease of our thoughts and the clarity of our feelings.

A compelling narrative fosters an illusion of inevitability. Consider the story of how Google turned into a giant of the technology industry. Two creative graduate students in the computer science department at Stanford University come up with a superior way of searching information on the Internet. They seek and obtain funding to start a company and make a series of decisions that work out well. Within a few years, the company they started is one of the most valuable stocks in America, and the two former graduate students are among the richest people on the planet. On one memorable occasion, they were lucky, which makes the story even more compelling: a year after founding Google, they were willing to sell their company for less than $1 million, but the buyer said the price was too high. Mentioning the single lucky incident actually makes it easier to underestimate the multitude of ways in which luck affected the outcome.

A detailed history would specify the decisions of Google’s founders, but for our purposes it suffices to say that almost every choice they made had a good outcome. A more complete narrative would describe the actions of the firms that Google defeated. The hapless competitors would appear to be blind, slow, and altogether inadequate in dealing with the threat that eventually overwhelmed them.

I intentionally told this tale blandly, but you get the idea: there is a very good story here. Fleshed out in more detail, the story could give you the sense that you understand what made Google succeed; it would also make you feel that you have learned a valuable general lesson about what makes businesses succeed. Unfortunately, there is good reason to believe that your sense of understanding and learning from the Google story is largely illusory. One check on such a lesson is whether it would have helped distinguish likely successes from failures before the outcomes were known. A retrospective story about Google cannot establish that its success was predictable: it leaves out many events that could have led elsewhere. The human mind does not deal well with nonevents. The fact that many of the important events that did occur involve choices further tempts you to exaggerate the role of skill and underestimate the part that luck played in the outcome. Because every critical decision turned out well, the record suggests almost flawless prescience—but bad luck could have disrupted any one of the successful steps. The halo effect adds the final touches, lending an aura of invincibility to the heroes of the story.

Like watching a skilled rafter avoiding one potential calamity after another as he goes down the rapids, the unfolding of the Google story is thrilling because of the constant risk of disaster. However, there is an instructive difference between the two cases. The skilled rafter has gone down rapids hundreds of times. He has learned to read the roiling water in front of him and to anticipate obstacles. He has learned to make the tiny adjustments of posture that keep him upright. There are fewer opportunities for young men to learn how to create a giant company, and fewer chances to avoid hidden rocks—such as a brilliant innovation by a competing firm. Of course there was a great deal of skill in the Google story, but luck played a more important role in the actual event than it does in the telling of it. And the more luck was involved, the less there is to be learned.

At work here is that powerful WYSIATI rule. You cannot help dealing with the limited information you have as if it were all there is to know. You build the best possible story from the information available to you, and if it is a good story, you believe it. Paradoxically, it is easier to construct a coherent story when you know little, when there are fewer pieces to fit into the puzzle. Our comforting conviction that the world makes sense rests on a secure foundation: our almost unlimited ability to ignore our ignorance.

The narrative fallacy operates automatically and often escapes our notice. Taleb recounts an anecdote about an Italian professor who was an enthusiastic supporter of Taleb’s ideas on the overestimation of cause and effect. The professor cornered Taleb at a conference and declared: “You show how stupid we are to systematically try to explain skills.” But then, in a calmer tone, he added: “Mon cher ami, had you grown up in a Protestant society where people are told that efforts are linked to rewards and individual responsibility is emphasized, you would never have seen the world in such a manner. You were able to see luck and separate cause and effect because of your Eastern Orthodox Mediterranean heritage.” In that single sentence, the professor—who moments earlier had railed against overinterpretation—could not help but invent a cause, a neat story, to explain Taleb’s work. Neither of them immediately recognized the irony. We are so vulnerable to overinterpretation that we weave stories even about why someone rejects stories.

The illusion of understanding is especially dangerous when we look back at rare, high-impact events—what Taleb calls Black Swans. In domains where outcomes are dominated by a few extreme winners (Extremistan), such as intellectual, scientific, or artistic activities, past narratives seem to work deceptively well. In contrast, in more regular, repeatable environments (Mediocristan), the past is more amenable to genuine learning. But in Extremistan, we must remain deeply suspicious of the easy and obvious narrative. We tend to forget about the notion of Black Swans immediately after one occurs—they are too abstract—and focus instead on the precise and vivid events that easily come to mind. We do worry about Black Swans, just the wrong ones.

I have heard of too many people who “knew well before it happened that the 2008 financial crisis was inevitable.” This sentence contains a highly objectionable word, which should be removed from our vocabulary in discussions of major events. The word is, of course, knew. Some people thought well in advance that there would be a crisis, but they did not know it. They now say they knew it because the crisis did in fact happen. This is a misuse of an important concept. In everyday language, we apply the word know only when what was known is true and can be shown to be true. We can know something only if it is both true and knowable. But the people who thought there would be a crisis (and there are fewer of them than now remember thinking it) could not conclusively show it at the time. Many intelligent and well-informed people were keenly interested in the future of the economy and did not believe a catastrophe was imminent. Their disagreement illustrates uncertainty at the time; it does not by itself establish that nobody could have identified the danger. What is perverse about the use of know in this context is not that some individuals get credit for prescience that they do not deserve. It is that the language implies that the world is more knowable than it is. It helps perpetuate a pernicious illusion.

The core of the illusion is that we believe we understand the past, which implies that the future also should be knowable, but in fact we understand the past less than we believe we do. Know is not the only word that fosters this illusion. In common usage, the words intuition and premonition also are reserved for past thoughts that turned out to be true. The statement “I had a premonition that the marriage would not last, but I was wrong” sounds odd, as does any sentence about an intuition that turned out to be false. To think clearly about the future, we need to clean up the language that we use in labeling the beliefs we had in the past.

To check an explanatory story, look for evidence that could distinguish it from alternatives. Record predictions before outcomes are known, and keep track of failures as well as successes. Experiments can test some claims; documents and archaeological evidence can test others. A narrative can communicate these findings, but its clarity is not itself evidence that it is right. The claim is “I knew it all along.” Compare the claim with what you actually recorded at the time.