Skip to main content
Decision Frameworks

Show Me the Evidence

A long reference list can make promotional claims look more convincing than their evidence warrants. The source examines a report on chess and intelligence, asking what kinds of research its references actually contain.

THE IDEA

Previous-Section Takeaways

  • ► You don’t always realize what you don’t know, so “zooming out” to get a summary of expert opinions is a good first step. Before diving headfirst into a new topic, start off by “reading the room.”
  • ► Cynicism doesn’t make you any smarter than gullibility. To avoid “trust compression,” distinguish between little problems and giant failings in the sources you evaluate.
  • ► Avoid relying on a single self-proclaimed expert. Instead, try to assess and understand the range of credible expert opinion on a subject.

6 Show Me the Evidence

Scholarly Sources and Promotional Claims

Can playing chess make you smarter? Why not? The game demands concentration, forethought, and cool-headed logic—the same virtues that define critical thinking.

Before you reach for that dusty chessboard, let’s look at the evidence.

When you ask Google about the connection between chess and intelligence, one of the first results to come up is a 104-page document called The Benefits of Chess in Education, compiled by Patrick S. McDonald, the youth coordinator for the Chess Federation of Canada. McDonald’s report claims that the movement of knights and rooks not only “makes kids smart” and “improves academic performance” but also “enhance[s] creativity, problem solving, memory, concentration, intellectual maturity, self-esteem, and many other abilities.”1 A veritable jackpot of benefits.

OK, clearly this author has a lot of skin in this game and may not be the most objective source. But the report backs up its claims with a staggering list of references—titles like “Scientific Proof: Chess Improves Reading Scores,” “Utilizing Chess to Promote Self-Esteem in Perceptually Impaired Students,” and “The Effect of Chess on Reading Scores: District Nine Chess Program Second Year Report.”

When you look a bit closer, however, an unmistakable pattern emerges. Most of the references cite magazine articles (including one that itself needs dusting off—from a 1931 National Geographic), along with newsletters from organizations like the “Commission of Chess in Schools,” scattered doctoral dissertations from the 1990s, and a mishmash of presentations delivered at random conferences. What ties them all together is not something they possess but something they lack. Almost all these references do an end run around the process known as peer review.

When to use it

  • Evaluating health claims from news articles or advertisements.

  • Researching a new diet or exercise program.

  • Assessing the credibility of scientific studies quoted in a debate or social media post.

  • Making decisions based on expert advice when choosing a course of action.

FROM THEORY TO EVERYDAY LIFE

In practice

01

Chess and Intelligence Claims

A student researching the benefits of chess for cognitive development finds a report by the Chess Federation of Canada that claims numerous benefits. Upon closer inspection, the student discovers that most of the cited sources are newsletters, conference presentations, and a 1931 National Geographic article, none of which are peer-reviewed research.

02

Trust Compression in Source Evaluation

A reader examines an article about a new diet. They distinguish a minor typo from a serious problem such as citing research that did not test the article’s central claim. They inspect the underlying evidence before accepting or dismissing the article as a whole.

✳

A MOMENT TO REFLECT

Put it to work.

  1. When you encounter a claim like 'chess makes you smarter', what steps can you take to verify the evidence?

  2. How can you distinguish between peer-reviewed research and promotional material in everyday reading?

  3. What are some signs that a source might be biased or lack credibility, beyond just looking at the publication venue?

CONTINUE THE CONVERSATION

Source texts

  • Verified How to Think Straight, Get Duped Less, and Make Better Decisions about What to Believe Online (Mike Caulfield, Sam Wineburg)
← More decision frameworks