Observational Selection Effect
Observational selection refers to the cognitive bias where we draw conclusions based only on visible evidence while ignoring hidden evidence that would contradict our beliefs. This leads to overestimating success and underestimating the role of chance.
THE IDEA
Observational selection occurs when we draw conclusions from the part of the picture we can see while overlooking what is missing. Nassim Taleb’s “Black Swan” uses a story attributed to Cicero. The Greek philosopher Diago, nicknamed the Atheist, was shown images of people who had prayed and survived a shipwreck. The implication was that prayer had saved them. Diago asked: “Where are the images of those who prayed, but still drowned?” The dead cannot provide matching testimonials. Looking only at the survivors hides evidence needed to assess the claim.
The excerpt calls another Taleb work "Fooled by Accident"; the reference list identifies it as Fooled by Randomness. Its discussion challenges the idea that success comes entirely from personal qualities and talent, overlooking chance and "hidden witnesses." Consider a group whose wealth comes from secretly playing Russian roulette. They spin the cylinder of a six-shot revolver containing one cartridge, pull the trigger, and receive money if they survive. About 1/6 of the group is eliminated each round. After several rounds, a small wealthy group remains, alongside a large cemetery. Can the “success” of the survivors be attributed to their talent?
Outsiders see the wealth but may not see the process that produced it or the risks taken. As the excerpt puts it, “no one is watching the bulletproof drum of reality. The naked eye very rarely sees a wealth generator. We observe the values produced, but never the processor (and this makes people lose sight of their risks) and never see the losers. The game seems awfully simple, and we blithely play it further.”
When to use it
Evaluating investment strategies by looking at both successful and failed funds, not just the winners.
Assessing a medical treatment's effectiveness by considering all patients, including those who did not improve or dropped out.
Analyzing business strategies by studying both successful and failed companies to avoid survivorship bias.
Understanding historical events by considering hidden evidence from those who did not survive to write history.
FROM THEORY TO EVERYDAY LIFE
In practice
Diago and the Shipwreck Images
Greek philosopher Diago was shown images of people who prayed and were saved from a shipwreck. He asked where the images were of those who prayed but drowned, highlighting hidden evidence that contradicts the conclusion that prayer leads to survival.
Russian Roulette Success
Taleb's thought experiment about a group playing Russian roulette: survivors become wealthy but we ignore the cemetery of those who died, illustrating that success may be due to luck rather than skill or talent.
A MOMENT TO REFLECT
Put it to work.
When you see a successful person, consider what hidden evidence of failures you might be missing. How does this affect your judgment of their skills versus luck?
In what ways might you be ignoring hidden evidence in your daily decisions, such as when evaluating a business or personal outcome?
Can you identify a situation where you assumed a cause based on visible outcomes without checking the unseen? How would your conclusion change with full evidence?
References
- The Black Swan — Nassim Nicholas Taleb (2007)
- Fooled by Randomness — Nassim Nicholas Taleb (2001)
CONTINUE THE CONVERSATION
Source texts
- Thinking in Systems and Mental Models Think Like a Super Thinker (Dawson, Marcus P)