Distributions
Distributions describe how a set of data points spreads out, helping to contextualize what to expect, predict probabilities, and understand the frequency of events. Different distribution shapes, like normal and power law, lead to very different real-world behaviors and risks.
THE IDEA
Distributions describe the values a variable can take and how frequently, or with what probability, those values occur. They can summarize observed data or serve as models for uncertain outcomes. A fitted model needs checking before it is used to predict future events.
Types of Distributions
Useful questions include:
- Are values discrete or continuous?
- Is the distribution symmetric or asymmetric?
- Are there upper or lower bounds?
- How much probability lies in extreme values?
As the account quotes NYU Stern School of Business: “Raw data is almost never as well behaved as we would like it to be. Consequently, fitting a statistical distribution to data is part art and part science, requiring compromises along the way. The key to good data analysis is maintaining a balance between getting a good distributional fit and preserving ease of estimation, keeping in mind that the ultimate objective is that the analysis should lead to better decisions.”
The Normal Distribution
A normal distribution has a symmetric bell-shaped density. Its mean, median, and mode coincide. Mlodinow writes: “The normal distribution describes the manner in which many phenomena vary around a central value that represents their most probable outcome.”
Some measurements can be approximated by a normal distribution over a useful range. That is a modeling claim to check, rather than a consequence of being a biological measurement, a test score, or a household price. A normal model extends without bounds in both directions, even when the quantity being modeled has physical limits.
Normal tails become thin rapidly. A process with frequent large delays or extreme outcomes may require a different model. An ordinary-looking center does not establish that the tails are normal.
Power Law Distributions
A power-law tail decreases according to a power of the quantity being measured. It is different from an exponential tail. Such a model can assign much greater probability to extreme values than a normal model does.
Wealth illustrates why tail behavior matters: people with $1,000 and people with $1 billion occupy very different scales. This contrast alone does not prove that an entire wealth distribution follows a power law. Christian and Griffiths use the term “scale-free distributions” for power laws because their form relates values across scales.
Waiting-time predictions also depend on the model. Under some heavy-tailed duration models, surviving longer can imply a longer remaining wait. That is a conditional property to derive from the distribution, rather than a rule for every process with an unusually long duration.
Other Distributions
- A geometric distribution models the number of independent trials needed for a success when the success probability stays constant.
- A binomial distribution models the number of successes in a fixed number of independent trials with a common success probability.
- A Poisson distribution models event counts under suitable assumptions; a common example is arrivals over a fixed interval at a constant average rate.
- A memoryless waiting-time model says that the remaining wait has the same distribution regardless of the time already spent waiting. Exponential waiting times have this property. A scheduled bus service need not.
Choose a model for the question being asked, check its assumptions against the data, and pay particular attention to the outcomes that would make a wrong model costly.
When to use it
A retailer compares candidate models with observed demand, checking seasonality, stockouts, and extreme orders before choosing inventory levels.
An investor assessing portfolio risk: financial returns frequently exhibit fat tails, not a perfect normal distribution, meaning extreme losses (or gains) are more likely than a bell curve would predict.
A teacher grading on a curve: assuming test scores are normally distributed, but actual student performance might be skewed, requiring careful consideration before applying a bell curve to grades.
A transport planner checks schedules and observed arrival gaps before deciding whether a memoryless waiting-time model is appropriate.
FROM THEORY TO EVERYDAY LIFE
In practice
Commuting Time and Long Tails
A commute may usually take a similar amount of time but occasionally be delayed by a closure or breakdown. Check how often large delays occur before treating the whole distribution as normal; a good fit near the center can conceal a poor fit in the tail.
Wealth as a Power Law
A wealth dataset may be strongly skewed, with a small number of very large values. A power law may describe part of its upper tail, but that needs to be tested rather than inferred from the existence of wealthy outliers.
Toothpaste Prices
Compare toothpaste prices for similar package sizes and products. Their average can provide a reference, but neither the usefulness of that reference nor the shape of the price distribution establishes that prices are normally distributed.
A MOMENT TO REFLECT
Put it to work.
Think about a metric you track regularly (e.g., your commute time, sales numbers, body weight). Does it follow a normal distribution, a power law, or something else? How would recognizing the correct distribution change your expectations or plans?
When have you assumed a normal distribution where a power law was more appropriate? What were the consequences of that mismatch?
How can understanding long tails help you prepare for unlikely but highly impactful events in your personal or professional life?
Often confused with
References
- The Drunkard's Walk: How Randomness Rules Our Lives — Leonard Mlodinow (2008)
- Algorithms to Live By: The Computer Science of Human Decisions — Brian Christian and Tom Griffiths (2016)
CONTINUE THE CONVERSATION
Source texts
- The Great Mental Models Volume 3 Systems and Mathematics