Silent Evidence (Survivorship Bias)

THE STORY OF THE DROWNED WORSHIPPERS

More than two thousand years ago, the Roman orator, belletrist, thinker, Stoic, manipulator-politician, and (usually) virtuous gentleman, Marcus Tullius Cicero, presented the following story. One Diagoras, a nonbeliever in the gods, was shown painted tablets bearing the portraits of some worshippers who prayed, then survived a subsequent shipwreck. The implication was that praying protects you from drowning. Diagoras asked, “Where were the pictures of those who prayed, then drowned?”

The drowned worshippers, being dead, would have a lot of trouble advertising their experiences from the bottom of the sea. This can fool the casual observer into believing in miracles.

We call this the problem of silent evidence. The idea is simple, yet potent and universal. While most thinkers try to put to shame those who came before them, Cicero puts to shame almost all empirical thinkers who came after him, until very recently.

Later on, both my hero of heroes, the essayist Michel de Montaigne and the empirical Francis Bacon, mentioned the point in their works, applying it to the formation of false beliefs. “And such is the way of all superstition, whether in astrology, dreams, omens, divine judgments, or the like,” wrote Bacon in his Novum Organum. The problem, of course, is that unless they are drilled into us systematically, or integrated into our way of thinking, these great observations are rapidly forgotten.

Silent evidence pervades everything connected to the notion of history. By history, I don’t just mean those learned-but-dull books in the history section (with Renaissance paintings on their cover to attract buyers). History, I will repeat, is any succession of events seen with the effect of posteriority.

This bias extends to the ascription of factors in the success of ideas and religions, to the illusion of skill in many professions, to success in artistic occupations, to the nature versus nurture debate, to mistakes in using evidence in the court of law, to illusions about the “logic” of history—and of course, most severely, in our perception of the nature of extreme events.

You are in a classroom listening to someone self-important, dignified, and ponderous (but dull), wearing a tweed jacket (white shirt, polka-dot tie), pontificating for two hours on the theories of history. You are too paralyzed by boredom to understand what on earth he is talking about, but you hear the names of big guns: Hegel, Fichte, Marx, Proudhon, Plato, Herodotus, Ibn Khaldoun, Toynbee, Spengler, Michelet, Carr, Bloch, Fukuyama, Schmukuyama, Trukuyama. He seems deep and knowledgeable, making sure that no attention lapse will make you forget that his approach is “post-Marxist,” “postdialectical,” or post-something, whatever that means. Then you realize that a large part of what he is saying reposes on a simple optical illusion! But this will not make a difference: he is so invested in it that if you questioned his method he would react by throwing even more names at you.

It is so easy to avoid looking at the cemetery while concocting historical theories. But this is not just a problem with history. It is a problem with the way we construct samples and gather evidence in every domain. We shall call this distortion a bias, i.e., the difference between what you see and what is there. By bias I mean a systematic error consistently showing a more positive, or negative, effect from the phenomenon, like a scale that unfailingly shows you a few pounds heavier or lighter than your true weight, or a video camera that adds a few sizes to your waistline. This bias has been rediscovered here and there throughout the past century across disciplines, often to be rapidly forgotten (like Cicero’s insight). As drowned worshippers do not write histories of their experiences (it is better to be alive for that), so it is with the losers in history, whether people or ideas. Remarkably, historians and other scholars in the humanities who need to understand silent evidence the most do not seem to have a name for it (and I looked hard). As for journalists, fuhgedaboudit! They are industrial producers of the distortion.

The term bias also indicates the condition’s potentially quantifiable nature: you may be able to calculate the distortion, and to correct for it by taking into account both the dead and the living, instead of only the living.

Silent evidence is what events use to conceal their own randomness, particularly the Black Swan type of randomness.

Sir Francis Bacon is an interesting and endearing fellow in many respects.

He harbored a deep-seated, skeptical, nonacademic, antidogmatic, and obsessively empirical nature, which, to someone skeptical, nonacademic, antidogmatic, and obsessively empirical, like this author, is a quality almost impossible to find in the thinking business. (Anyone can be skeptical; any scientist can be overly empirical—it is the rigor coming from the combination of skepticism and empiricism that’s hard to come by.) The problem is that his empiricism wanted us to confirm, not disconfirm; thus he introduced the problem of confirmation, that beastly corroboration that generates the Black Swan.