Fallacy of the Single Event Probability

FALLACY OF THE SINGLE EVENT PROBABILITY

As the life-expectancy example in Chapter 10 shows, the conditional expectation of additional life drops as one advances in age (as you get older you are expected to live a smaller number of years; this comes from the fact that there is an asymptotic “soft” ceiling to how old a human can get). Expressing it in units of standard deviations, the conditional expectation of a Mediocristani Gaussian variable, conditional on it being higher than a threshold of 0, is .8 (standard deviations). Conditional on it being higher than a threshold of 1, it will be 1.52. Conditional on it being higher than 2, it will be 2.37. As you see, the two numbers should converge to each other as the deviations become large, so conditional on it being higher than 10 standard deviations, a random variable will be expected to be just 10.

In Extremistan, things work differently. The conditional expectation of an increase in a random variable does not converge to the threshold as the variable gets larger. In the real world, say with stock returns (and all economic variables), conditional on a loss being worse than 5 units, using any unit of measure (it makes little difference), it will be around 8 units. Conditional that a move is more than 50 units, it should be around 80 units, and if we go all the way until the sample is depleted, the average move worse than 100 units is 250 units! This extends to all areas in which I found sufficient samples. This tells us that there is “no” typical failure and “no” typical success. You may be able to predict the occurrence of a war, but you will not be able to gauge its effect! Conditional on a war killing more than 5 million people, it should kill around 10 million (or more). Conditional on it killing more than 500 million, it would kill a billion (or more, we don’t know). You may correctly predict that a skilled person will get “rich,” but, conditional on his making it, his wealth can reach $1 million, $10 million, $1 billion, $10 billion—there is no typical number. We have data, for instance, for predictions of drug sales, conditional on getting things right. Sales estimates are totally uncorrelated to actual sales—some drugs that were correctly predicted to be successful had their sales underestimated by up to 22 times.

This absence of “typical” events in Extremistan is what makes something called prediction markets (in which people are assumed to make bets on events) ludicrous, as they consider events to be binary. “A war” is meaningless: you need to estimate its damage—and no damage is typical. Many predicted that the First World War would occur, but nobody really predicted its magnitude. One of the reasons economics does not work is that the literature is almost completely blind to this point.

Accordingly, Ferguson’s methodology (mentioned in Chapter 1) in looking at the prediction of events as expressed in the price of war bonds is sounder than simply counting predictions, because a bond, reflecting the costs to the governments involved in a war, is priced to cover the probability of an event times its consequences, not just the probability of an event. So we should not focus on whether someone “predicted” an event without his statement having consequences attached to it.

Associated with the previous fallacy is the mistake of thinking that my message is that these Black Swans are necessarily more probable than assumed by conventional methods. They are mostly less probable, but have bigger effects. Consider that, in a winner-take-all environment, such as the arts, the odds of success are low, since there are fewer successful people, but the payoff is disproportionately high. So, in a fat-tailed environment, rare events can be less frequent (their probability is lower), but they are so powerful that their contribution to the total pie is more substantial.

The point is mathematically simple, but does not register easily. I’ve enjoyed giving graduate students in mathematics the following quiz (to be answered intuitively, on the spot). In a Gaussian world, the probability of exceeding one standard deviation is around 16 percent. What are the odds of exceeding it under a distribution of fatter tails (with the same mean and variance)? The right answer: lower, not higher—the number of deviations drops, but the few that take place matter more. It was puzzling to see that most graduate students get it wrong.

Back to stress testing again. At the time this account was written, the U.S. government was having financial institutions stress-tested by assuming large deviations and checking the results against the capitalization of these firms. But the problem is, Where did they get the numbers? From the past? This is so flawed, since the past, as we saw, is no indication of future deviations in Extremistan. This comes from the atypicality of extreme deviations. My experience of stress testing is that it reveals little about the risks—but the risks can be used to assess the degree of model error.