Prosecutor's Fallacy
The prosecutor's fallacy is a logical fallacy that occurs when the probability of a false match (given innocence) is confused with the probability of innocence (given a match). This leads to overestimating the likelihood of guilt in legal contexts and misinterpreting p-values in scientific studies.
THE IDEA
The Prosecutor's Fallacy
The prosecutor's fallacy is a common error in reasoning about conditional probabilities. It arises when one confuses P(match|innocent) with P(innocent|match). This fallacy is illustrated by a courtroom example involving fingerprint evidence.
In the story, a defendant is identified by a fingerprint match with a one-in-ten-million false positive rate. The prosecutor argues that there is only a one-in-ten-million chance that the defendant is innocent. However, the defense attorney correctly points out that this is P(match|innocent), not P(innocent|match). In a simplified database model of 50 million people that includes the culprit, expect about 5 false matches and 1 true match if the culprit’s print is detected. Treating a match as otherwise indistinguishable gives an approximate innocence probability of 5/6. That illustration requires these assumptions and is not a posterior justified by database size alone.
This fallacy is also known as the base rate fallacy when it involves ignoring the base rate of a condition. In medical testing, a positive test for a rare disease is more likely to be a false positive if the test is not extremely accurate.
Connection to p-values
Scientific p-values are often misinterpreted in a similar way. A p-value of 0.05 means that if the null hypothesis were true, there is a 5% chance of observing data at least as extreme. This is a tail probability under the null model, often abbreviated as P(data|null); it is not P(null|data). Yet scientists and journalists often treat p < 0.05 as evidence that the null hypothesis is false with 95% confidence. The prosecutor's fallacy is a key reason why many published research findings may be false, as argued by Ioannidis (2005).
When to use it
Evaluating the strength of DNA evidence in criminal trials
Interpreting positive results in medical screening tests
Assessing the credibility of scientific findings reported in the media
Understanding the limitations of p-values in research studies
FROM THEORY TO EVERYDAY LIFE
In practice
Fingerprint match in court
The hypothetical fingerprint search has a one-in-ten-million false-positive rate, which the prosecutor mistakes for a one-in-ten-million innocence probability. With 50 million database entries, expect about 5 false matches. An approximate 5/6 innocence probability additionally assumes a detected culprit in that database and no evidence distinguishing the matches.
Lyme disease test
In a hypothetical Lyme screening example, assume a 5% false-positive rate and 0.1% prevalence. Among 10,000 people, expect 10 infected people and about 500 false positives. Even if every infection is detected, fewer than 2% of positive results would be true positives under these assumptions.
A MOMENT TO REFLECT
Put it to work.
Think of a news article that reports a scientific study with a low p-value. How might the prosecutor's fallacy affect the interpretation of that study?
When evaluating legal evidence, why is it important to consider the size of the database from which a match was found?
How can understanding the prosecutor's fallacy help you avoid being misled by medical test results?
Often confused with
References
- Why Most Published Research Findings Are False — John Ioannidis (2005)
CONTINUE THE CONVERSATION
Source texts
- Calling Bullshit The Art of Skepticism in a Data-Driven World (Carl T. Bergstrom Jevin D. West)