Numbers That Bamboozle
Statistics and numbers can be deeply misleading if you don't understand their origin or how they were generated. The case of VAERS data being misrepresented as evidence of vaccine deaths illustrates how context and data provenance are critical for interpretation.
THE IDEA
Numbers That Bamboozle
When you think of cheap signals, numbers and statistics don't immediately come to mind. Numbers are the hard evidence that back up a claim, right? But if you have no idea where the numbers come from or how they were generated, they're as useful as a bikini on an excursion to the Antarctic.
Take this story from The Defender, the online newsletter of Robert F. Kennedy Jr.'s Children's Health Defense organization. Lacking a degree in medicine but trading on his family's famous name, Kennedy has emerged as a leading spokesman for the antivaccine movement. His website recycles debunked claims tying vaccines to autism.17 As the pandemic surged, his antivaccine mission saw its site's gross revenue swell from $1.1 million in 2018 to $6.8 million in 2020.18 The site broadcasts bone-chilling numbers, as seen in this headline:
Figure 2.6 Misleading headline from the newsletter of Children's Health Defense
Thousands of deaths caused by the COVID vaccine? And what is "VAERS data"? Statistics provided by some science-denying, conspiracy-mongering, fear-generating website?
Actually, no.
VAERS (Vaccine Adverse Event Reporting System) is a US government database tracking vaccine safety. VAERS is jointly administered by the Centers for Disease Control and Prevention (CDC), the chief governmental body responsible for dispensing information about public health, and the Food and Drug Administration (FDA), the agency responsible for approving new drugs.
Doctors, nurses, and other health-care professionals can submit reports to VAERS. But so can you and your unhinged next-door neighbor. VAERS is a come-as-you-are, no-ticket-required database. Anyone and everyone can upload a report—and they do. Originally designed as an early warning system to capture adverse vaccine reactions that might have been missed during clinical trials, VAERS was created to gain insight into what ordinary people, not just health professionals, thought was happening. But ordinary people have some bizarro ideas. Take this example: Fred got vaccinated on Tuesday and struck by lightning on Wednesday. If he believed the two were related (maybe the vaccine increased his ability to conduct electricity?), he would be within his right to file an "adverse reaction" report.
The VAERS website publishes plenty of disclaimers warning people that its numbers contain "incomplete, inaccurate, coincidental or unverifiable information."19 But that hasn't stopped groups like Kennedy's from weaponizing VAERS to their advantage. Prominent medical officials think the site has outlived its purpose and advocate shutting it down. To dramatize the site's problems, an anesthesiologist named James Laidler submitted a report after a flu vaccination saying that it caused his skin to turn green and his muscles to bulk, turning him into the Incredible Hulk.20 Only after the CDC personally appealed did Laidler agree to remove the report. "If I had not agreed," he wrote, "the record would be there still, showing that any claim can become part of the database, no matter how outrageous or improbable."21
When to use it
Evaluating health claims in news articles by examining the data sources and collection methods.
Assessing the credibility of statistics shared on social media, especially those that seem alarming.
Understanding the limitations of self-reported data in surveys and public databases.
Analyzing the integrity of statistical evidence in scientific research and policy debates.
FROM THEORY TO EVERYDAY LIFE
In practice
Misuse of VAERS Data
Children’s Health Defense presented VAERS reports as evidence of deaths caused by vaccination. The error is treating reports of events after vaccination as established causal findings. Reports can help identify possible safety signals for investigation, but a report alone does not establish that the vaccine caused the event.
A MOMENT TO REFLECT
Put it to work.
What questions should you ask about a data source before trusting a statistic derived from it?
How might someone deliberately misuse a passive reporting system like VAERS to spread misinformation?
What steps can you take to verify the reliability of statistical claims in news articles?
CONTINUE THE CONVERSATION
Source texts
- Verified How to Think Straight, Get Duped Less, and Make Better Decisions about What to Believe Online (Mike Caulfield, Sam Wineburg)