Biased Sample Fallacy
Drawing a conclusion about a population based on a sample that is biased or selected to misrepresent the true average, often leading to inaccurate generalizations.
THE IDEA
Description
(also known as: biased statistics, loaded sample, prejudiced statistics, prejudiced sample, loaded statistics, biased induction, biased generalization, unrepresentative sample, unrepresentative generalization)
Drawing a conclusion about a population based on a sample that is biased, or chosen in order to make it appear the population on average is different than it actually is. This differs from the hasty generalization fallacy, where the biased sample is specifically chosen from a select group, and the small sample is just a random sample, but too small to get any accurate information.
Logical Form
- Premise 1: A sample of a population is taken.
- Premise 2: The sample is biased (not representative of the population).
- Conclusion: A generalization about the population is drawn based on that sample.
Examples
Example #1
A survey of homeowners in Beverly Hills, CA shows that Americans are extremely wealthy.
Explanation: Where did these homeowners live? Beverly Hills, CA. If the same exact survey was taken in Detroit, the results would be quite different. It is fallacious to accept the conclusion about the American population in general based on not just the geographical sample, but also the fact that homeowners were only surveyed.
Example #2
Pastor Pete interviewed people in his church and concluded that everyone believes in God.
Explanation: Pastor Pete has drawn a conclusion about religious beliefs from people “everywhere” based on people he has interviewed in his church. That’s like concluding that the world likes to dance naked in front of strangers after interviewing a group of strippers.
Exception
What exactly is “biased” is subjective, but some biases are very clear.
Tip
Be very weary of statistics. Look at the source and details of the studies which produced the statistics. Very often you will find some kind of bias.
When to use it
Evaluating opinion polls: A poll that only surveys people from a specific region or demographic may yield biased results about a broader population.
Market research: Using a sample of only existing customers to gauge overall satisfaction can overestimate positive sentiment.
Medical studies: A clinical trial with a non-diverse sample may produce biased conclusions about drug efficacy across populations.
Academic research: Generalizing findings from a convenience sample (e.g., college students) to the general public can commit this fallacy.
FROM THEORY TO EVERYDAY LIFE
In practice
Homeowner Survey in Beverly Hills
A survey of homeowners in Beverly Hills, CA shows that Americans are extremely wealthy. Explanation: Where did these homeowners live? Beverly Hills, CA. If the same exact survey was taken in Detroit, the results would be quite different. The sample excludes people outside Beverly Hills and people who do not own homes, so it cannot support a conclusion about Americans generally.
Pastor Pete's Church Interview
Pastor Pete interviewed people in his church and concluded that everyone believes in God. Explanation: Pastor Pete has drawn a conclusion about religious beliefs from people “everywhere” based on people he has interviewed in his church. That’s like concluding that the world likes to dance naked in front of strangers after interviewing a group of strippers.
A MOMENT TO REFLECT
Put it to work.
How can you ensure that the sample you use to draw a conclusion is representative of the population?
What are some common sources of bias in surveys or studies that lead to biased sample fallacies?
When you encounter a statistic, what questions should you ask about the sample to evaluate its validity?
Often confused with
CONTINUE THE CONVERSATION
Source texts
- Logically Fallacious - The Ultimate Collection of Over 300 Logical Fallacies