Skip to main content
Cognitive Biases

Survivorship Bias in Managerial Track Records

Even a population composed entirely of incompetent managers will produce a small number with great track records due to randomness. This makes it difficult to distinguish skill from luck based solely on past performance.

THE IDEA

Nobody Has to Be Competent

Let’s push the argument further to make it more interesting. We create a cohort that is composed exclusively of incompetent managers. We will define an incompetent manager as someone who has a negative expected return, the equivalent of the odds being stacked against him. We instruct the Monte Carlo generator now to draw from an urn. The urn has 100 balls, 45 black and 55 red. By drawing with replacement, the ratio of red to black balls will remain the same. If we draw a black ball, the manager will earn $10,000. If we draw a red ball, he will lose $10,000. The manager is thus expected to earn $10,000 with 45% probability, and lose $10,000 with 55%. On average, the manager will lose $1,000 each round—but only on average.

At the end of the first year, we still expect to have 4,500 managers turning a profit (45% of them), the second, 45% of that number, 2,025. The third, 911; the fourth, 410; the fifth, 184. Let us give the surviving managers names and dress them in business suits. True, they represent less than 2% of the original cohort. But they will get attention. Nobody will mention the other 98%. What can we conclude?

The first counterintuitive point is that a population entirely composed of bad managers will produce a small amount of great track records. As a matter of fact, assuming the manager shows up unsolicited at your door, it will be practically impossible to figure out whether he is good or bad. The results would not markedly change even if the population were composed entirely of managers who are expected in the long run to lose money. Why? Because owing to volatility, some of them will make money. We can see here that volatility actually helps bad investment decisions.

The second counterintuitive point is that the expectation of the maximum of track records, with which we are concerned, depends more on the size of the initial sample than on the individual odds per manager. In other words, the number of managers with great track records in a given market depends far more on the number of people who started in the investment business (in place of going to dental school), rather than on their ability to produce profits. It also depends on the volatility. Why do I use the notion of expectation of the maximum? Because I am not concerned at all with the average track record. I will get to see only the best of the managers, not all of the managers. This means that we would see more “excellent managers” in 2006 than in 1998, provided the cohort of beginners was greater in 2001 than it was in 1993—I can safely say that it was.

When to use it

  • Evaluating investment managers based on past performance

  • Assessing employee performance in large organizations

  • Interpreting scientific findings: many studies may produce false positives by chance

  • Analyzing sports statistics: a few players may have hot streaks due to randomness

FROM THEORY TO EVERYDAY LIFE

In practice

01

Incompetent Managers with Positive Track Records

Consider a cohort of 10,000 managers, each with a 45% chance of earning $10,000 and a 55% chance of losing $10,000 per year (expected loss of $1,000 per year). After five years, about 184 managers (1.84%) will have made money every year purely by chance. These managers will appear to be skilled, but they are actually part of the incompetent cohort.

✳

A MOMENT TO REFLECT

Put it to work.

  1. How can you tell if a fund manager's consistent returns are due to skill or just luck?

  2. In what other domains might a small number of lucky successes be mistaken for competence?

  3. How does the size of the initial cohort affect the number of apparent winners?

Connected ideas

Another lens can bring the same problem into focus.

Often confused with

CONTINUE THE CONVERSATION

Source texts

  • Fooled by Randomness The Hidden Role of Chance in the Markets and in Life (Nassim Nicholas Taleb)
← More cognitive biases