Vicious Bias
A bias where the most lethal risks become invisible because severely affected individuals are eliminated from the evidence, leading to a false impression that a harmful treatment is strengthening.
THE IDEA
Vicious Bias
There is a vicious attribute to the bias: it can hide best when its impact is largest. Owing to the invisibility of the dead rats, the more lethal the risks, the less visible they will be, since the severely victimized are likely to be eliminated from the evidence. The more injurious the treatment, the larger the difference between the surviving rats and the rest, and the more fooled you will be about the strengthening effect. One of the two following ingredients is necessary for this difference between the true effect (weakening) and the observed one (strengthening): a) a degree of inequality in strength, or diversity, in the base cohort, or b) unevenness, or diversity, somewhere in the treatment. Diversity here has to do with the degree of uncertainty inherent in the process.
When to use it
Evaluating the effectiveness of a drug when dropouts due to severe side effects are removed from the analysis.
Assessing the success rate of entrepreneurs by only looking at those who are still in business, ignoring those who failed.
Analyzing military aircraft armor placement based on returning planes, which ignores planes that were shot down and never returned.
Interpreting historical survival rates of a disease when only the healthiest patients are still alive at the time of data collection.
FROM THEORY TO EVERYDAY LIFE
In practice
Rat treatment study
In a hypothetical study of a treatment’s effect on rats, those that are more severely affected die and are removed from the sample, so the surviving rats appear stronger, giving the illusion that the treatment is strengthening.
A MOMENT TO REFLECT
Put it to work.
How might this bias affect your perception of successful strategies or treatments?
Can you think of a situation where the most severe failures are hidden from view, leading to overly optimistic conclusions?
What steps can be taken to account for the invisible victims in your data or analysis?
CONTINUE THE CONVERSATION
Source texts
- The Black Swan The Impact of the Highly Improbable With a new section On Robustness and Fragility (Nassim Nicholas Taleb)