Quantifying Your Uncertainty
Quantifying uncertainty means expressing confidence as a probability and checking it against later outcomes. Calibration improves when claims assigned a given confidence level prove correct at roughly that rate over many cases.
THE IDEA
QUANTIFYING YOUR UNCERTAINTY
Typically, when people think about how sure they are, they ask themselves something like: “Do I actively feel any doubt?” If the answer is no, as it often is, they declare themselves to be “100% certain.”
That’s an understandable way to think about certainty, but it’s not the way a scout thinks about it. A scout treats their degree of certainty as a prediction of their likelihood of being right. Imagine sorting all of your beliefs into buckets based on how sure you are that you’re right about each one. This would include quotidian predictions (“I will enjoy this restaurant”), beliefs about your life (“My partner is faithful to me”), beliefs about how the world works (“Smoking causes cancer”), core premises (“Magic isn’t real”), and so on. Putting a belief into the “70% sure” bucket is like saying, “This is the kind of thing I expect to get right roughly 70 percent of the time.”
What you’re implicitly aiming for when you tag your beliefs with various confidence levels is perfect calibration. That means your “50% sure” claims are in fact correct 50 percent of the time, your “60% sure” claims are correct 60 percent of the time, your “70% sure” claims are correct 70 percent of the time, and so on.
When to use it
When estimating project completion dates, assign confidence percentages to each milestone to communicate risk more accurately.
During investment decisions, use probability ranges instead of binary 'will go up/will go down' to better assess expected value.
When evaluating competing hypotheses in scientific research, quantify the likelihood of each to avoid overconfidence in a single explanation.
In personal relationships, applying probability estimates to assumptions (e.g., '90% sure my partner is happy') can prompt more open communication.
FROM THEORY TO EVERYDAY LIFE
In practice
Restaurant Enjoyment Prediction
You estimate a 70% chance you will enjoy a new restaurant. Over many such predictions, if you are well calibrated, you should enjoy about 70% of the restaurants you rate as 70% likely.
Weather Forecast Calibration
A meteorologist who says '70% chance of rain' should see rain on about 70% of such forecasts, not necessarily on the next specific day.
A MOMENT TO REFLECT
Put it to work.
Think of a belief you hold with 100% certainty. How would your decision-making change if you assigned a probability of 95% or 99% instead?
Over the next week, assign probabilities to at least three everyday predictions, such as arriving on time or liking a movie. Record the outcomes and continue building a larger set before judging your calibration.
In what areas of your life are you most likely to be overconfident? How could quantifying your uncertainty help reduce that bias?
CONTINUE THE CONVERSATION
Source texts
- The Scout Mindset (Julia Galef)