Multiple Model Bias
The tendency to focus on a single model that fits data, ignoring that many models may fit equally well when viewed on a broader scale.
THE IDEA
In data analysis and interpretation, we often select a model that fits the observed data well without considering whether other models might fit just as well or better at a broader scale. This can lead to overconfident conclusions and spurious correlations. As the source puts it, 'We look at a broader scale. Hey, other models also fit it rather well.'
When to use it
Evaluating statistical models in research
Making investment decisions based on backtesting
Interpreting historical cause-and-effect relationships
FROM THEORY TO EVERYDAY LIFE
In practice
Stock Market Predictions
An analyst finds a model that accurately predicts stock prices for past data, but when tested on new data, it fails because many other models also fit the past data.
A MOMENT TO REFLECT
Put it to work.
When have you assumed a single explanation for a pattern without considering alternatives?
How can you test whether your model is uniquely fitting the data or just one of many?
Often confused with
CONTINUE THE CONVERSATION
Source texts
- The Black Swan The Impact of the Highly Improbable With a new section On Robustness and Fragility (Nassim Nicholas Taleb)