A study of millionaires may find courage and risk-taking among its subjects. To learn whether those traits help predict success, researchers also need information about people with the same traits who did not become wealthy. Studying winners alone cannot separate skill, circumstances, and luck.

By firing the worst-performing fund managers each year and keeping only the winners, a purely random cohort can produce long-term winners that appear skilled. This is survivorship bias: we see the survivors but not the many who failed.

Accounts of Bill Gates and Mark Zuckerberg can make leaving college seem central to entrepreneurial success. A useful comparison includes other people who left college, including those whose businesses failed, and founders who completed their education.

Historical analyses often focus on victorious battles and successful generals, while ignoring the vast number of defeats and failed strategies. This can lead to an overly optimistic view of certain tactics or leadership styles.

In the familiar account of Abraham Wald’s wartime analysis, returning planes had many holes in some areas and fewer near the engines. Those observations describe damage a plane could survive. They do not directly reveal where damage brought down the missing aircraft.

Morningstar's Large Blend category showed an average 178.4% growth from 1995-2004, but this ignored the funds that failed or were liquidated during that period. A 2006 study by Savant Capital highlighted that survivorship bias made the average performance appear higher than it truly was.

Mutual funds that fail are often excluded from performance data, giving an inflated view of average returns. A 2011 study in the Review of Finance found that the excess return of survivors was about 20% higher than when including failed funds.

Wald’s warplane problem asks what the observed sample excludes. If planes hit in a critical area rarely return, that area may show few holes among survivors. The inference depends on assumptions about exposure to fire and the chances of returning after a hit.

The ranking attributed to Matthew White compares bloodlettings of the 20th century by the proportion of a population killed. Its top three are the Herero massacre, the Cambodian genocide under Pol Pot, and King Leopold’s Congo. A ranking by absolute deaths could look different. This concerns the choice of denominator rather than survivorship bias alone.

The account reports that the Rwandan genocide killed 75% of the Tutsi population, while the 9/11 attacks killed 0.001% of Americans. These percentages use different reference populations. A person directly exposed to an attack can be a survivor regardless of its death toll as a share of the national population.

The account scales two catastrophes to Switzerland: the Rwandan genocide is represented as killing 75% of the population, and the 9/11 attacks as killing 80 people. These illustrations depend on the population and date used. Equal national proportions do not capture every difference in exposure, suffering, or consequences.

A profile of self-made millionaires can explain their experiences without establishing how often the same choices lead to wealth. People who worked equally hard but achieved different results provide an important comparison.

Famous experts are selected partly because they succeeded and remained visible. To assess their guidance, look for comparable people who used the same approach and failed, and ask whether the advice identifies a repeatable skill.

Stories about Facebook or Airbnb describe exceptional outcomes. Estimating the chances of building such a company requires a cohort of new businesses, including those that closed, rather than a collection of famous founders.

Investors often evaluate mutual funds by looking at the top-performing funds from the past decade, not realizing that many other funds with similar strategies have been closed due to poor performance, creating a biased sample.

Taleb contrasts Kenny, who treats his traders’ bull-market profits as evidence of skill, with Jean-Patrice, who examines the process generating the results. Later losses challenge Kenny’s confidence. The example asks whether a favorable market and selective retention made skill appear more certain than it was.

Victorious generals such as Alexander the Great and Julius Caesar dominate familiar histories. To evaluate a strategy, compare it with similar approaches used in defeats. Victory alone does not reveal how much skill or luck contributed, and forgotten opponents cannot simply be assumed equally capable.

George Will criticized Robert Shiller’s warning about overpriced stocks because the market kept rising. Judging the warning solely by that subsequent movement is outcome bias. Survivorship bias would arise if the comparison also excluded forecasters or strategies that disappeared from view.

Profiles of successful founders are useful accounts of those businesses, but their selection favors interesting winners. Using that coverage to estimate a new venture’s prospects omits businesses that never attracted a profile.

Taleb’s character Nero Tulip understands probability yet feels pressure when comparing himself with visible successes. The story illustrates how knowing about selection bias does not necessarily remove its emotional effect.

Wealthy residents can afford Park Avenue, so their address does not establish that living there produces wealth. That is a reverse-causation problem. Comparing oneself only with residents of an unusually wealthy neighborhood adds a separate selection problem.

Marc is a highly successful Harvard and Yale graduate earning in the top 0.5% of the US population. Yet his wife Janet feels he is a failure because they live on Park Avenue among extremely wealthy neighbors—a sample that excludes failures. Their relative low status in that exclusive environment distorts her perception of his actual success.

Taleb criticizes conclusions about frugality drawn from existing millionaires. A stronger comparison would include frugal people who did not become wealthy and examine whether habits changed after wealth was acquired. The issue is what a sample of current millionaires can establish about causes.

A book profiling successful investors during the long bull market starting in 1982 ignores those who lost money or invested in underperforming assets, creating a misleading impression that the strategy is universally profitable.

Taleb points to Lebanese lira Treasury bills, Michael Milken’s junk bonds, Russian Imperial bonds, and Argentine real estate in the 1930s as cases that can disappear from success-centered investment histories. Such losses belong in the historical comparison; a collection of winners cannot represent every available choice.

A guru and his peer proposed a 'Robin Hood' strategy of investing with the least successful fund manager and switching from winners to losers. Their backtest showed superior returns, but the analysis was flawed because it only included managers who survived to the present. Managers who went out of business due to poor performance were excluded, so the sample was biased towards recovery stories. Proper simulation would include all managers from a start date, regardless of survival, to avoid survivorship bias.

Paul Wilmott managed to extract praise from a bad review by selecting only positive words, using it on his book jacket despite the review being unfavorable.

Sartre's remark that Dos Passos was 'the greatest writer of our time' was mistaken by European intellectuals as a consensus estimate, making Dos Passos required reading, even though it was merely an extreme opinion.

Stories about Steve Jobs or Elon Musk can help generate ideas, but estimating the odds of success requires a defined group of ventures followed from their start. Choosing the group after seeing who became famous excludes much of the relevant evidence.

The warplane example distinguishes two questions: where were surviving aircraft hit, and where would extra armor most improve survival? Returning aircraft answer the first directly. Answering the second also requires reasoning about the aircraft that did not return.