Survivorship and Other Biases
A winner selected from many random attempts may have an impressive record without repeatable skill. Taleb’s monkey example asks whether the selection process, including unseen failures, supports a prediction of future success.
THE IDEA
Survivorship and Other Biases
If an infinite number of monkeys type independently at random on (strongly built) typewriters, with every required character possible, then with probability one a monkey produces an exact version of the Iliad. Upon examination, this may be less interesting a concept than it appears at first: For any particular monkey in a specified run, the probability is ridiculously low. But let us carry the reasoning one step beyond. Now that we have found that hero among monkeys, would any reader invest his life’s savings on a bet that the monkey would write the Odyssey next?
In this thought experiment, it is the second step that is interesting. How much can past performance (here the typing of the Iliad) be relevant in forecasting future performance? The same applies to any decision based on past performance, merely relying on the attributes of the past time series. Think about the monkey showing up at your door with his impressive past performance. Hey, he wrote the Iliad.
The major problem with inference in general is that those whose profession is to derive conclusions from data often fall into the trap faster and more confidently than others. The more data we have, the more likely we are to drown in it. For common wisdom among people with a budding knowledge of probability laws is to base their decision making on the following principle: It is very unlikely for someone to perform considerably well in a consistent fashion without his doing something right. Track records therefore become preeminent. They call on the rule of the likelihood of such a successful run and tell themselves that if someone performed better than the rest in the past then there is a great chance of his performing better than the crowd in the future—and a very great one at that. But, as usual, beware the middlebrow: A small knowledge of probability can lead to worse results than no knowledge at all.
When to use it
Evaluating investment strategies by looking only at funds with high past returns, ignoring those that closed.
Analyzing startup success stories without considering the high failure rate.
Judging the effectiveness of a marketing campaign solely by the few viral successes.
Assessing sports players' performance by focusing on their best games.
FROM THEORY TO EVERYDAY LIFE
In practice
The Monkey and the Iliad
In the thought experiment, a monkey typing randomly eventually produces the Iliad. If we only see the successful monkey, we might wrongly believe that monkey is likely to produce the Odyssey next, ignoring the many failures that didn't produce anything.
A MOMENT TO REFLECT
Put it to work.
How can the monkey typewriter example help you avoid survivorship bias when evaluating the performance of mutual funds?
In what ways might data overload lead to worse decision-making, as hinted in the text?
Think of a real-world situation where you have been misled by focusing only on successful outcomes. What data did you ignore?
CONTINUE THE CONVERSATION
Source texts
- Fooled by Randomness The Hidden Role of Chance in the Markets and in Life (Nassim Nicholas Taleb)