Overconfidence in Speed of Light Measurements
The reported history of speed-of-light measurements from 1876 to 1973 illustrates uncertainty estimates that were too narrow: later values departed from earlier estimates more often than the stated uncertainties suggested.
THE IDEA
The speed of light (c) has been measured many times, with estimates changing over time. From 1876 to 1902, measurements overestimated c by about 70 km/s on average; from 1905 to 1950, they underestimated by about 15 km/s. These shifts prompted suggestions that c was changing (e.g., deBray suggested a decrease of 4 km/s/year; Edmondson proposed sinusoidal variation). Birge (1942) argued for a constant value, but his adjusted estimates still showed overconfidence, as later values shifted by 2.4 standard deviations in 1950 and by over 2 standard deviations after that. Various calibration statistics (Birge ratio, interquartile index, surprise index) quantitatively show overconfidence: only 41% of values fell between the 0.25 and 0.75 quartiles (instead of 50%), and 11% were surprises (instead of 2%).
When to use it
Forecasting project completion times and budgets
Estimating the value of a physical constant or experimental result
Setting confidence intervals in data analysis and prediction
Making probabilistic predictions in business or policy decisions
FROM THEORY TO EVERYDAY LIFE
In practice
Speed of Light Measurements (1876-1973)
From 1876 to 1902, measurements overestimated the speed of light by 70 km/s on average; from 1905 to 1950, they underestimated by 15 km/s. In 1941, Birge adjusted estimates and claimed the value had settled, but within 9 years the recommended value shifted by 2.4 of his 1941 standard deviations. Similarly, the 1950 value was soon supplanted by a value different by more than 2 of its standard deviations, indicating persistent overconfidence in uncertainty estimates.
A MOMENT TO REFLECT
Put it to work.
How might overconfidence in uncertainty estimates affect scientific progress? Give a personal example where you were overconfident in a prediction or estimate.
What steps can be taken to improve calibration and reduce overconfidence in your own judgments?
Think of a situation where narrow confidence intervals led to a surprise. What could have been done differently?
Often confused with
References
- — Birge, R.T. (1942)
CONTINUE THE CONVERSATION
Source texts
- Heuristics and Biases The Psychology of Intuitive Judgment (Thomas Gilovich, Dale Griffin, Daniel Kahneman)