Cherry-picking (or the Fallacy of Confirmation)

Cherry-picking (or the Fallacy of Confirmation)

Consider the tourist brochures used by countries to advertise their wares: you can expect that the pictures presented to you will look much, much better than anything you will encounter in the place. And the bias, the difference (for which humans correct, thanks to common sense), can be measured as the country shown in the tourist brochure minus the country seen with your naked eyes. That difference can be small, or large. We also make such corrections with commercial products, not overly trusting advertising.

But we don’t correct for the difference in science, medicine, and mathematics, for the same reasons we didn’t pay attention to iatrogenics. We are suckers for the sophisticated.

In institutional research, one can selectively report facts that confirm one’s story, without revealing facts that disprove it or don’t apply to it—so the public perception of science is biased into believing in the necessity of the highly conceptualized, crisp, and purified Harvardized methods. And statistical research tends to be marred with this one-sidedness. Another reason one should trust the disconfirmatory more than the confirmatory.

Academia is well equipped to tell us what it did for us, not what it did not—hence how indispensable its methods are. This ranges across many things in life. Traders talk about their successes, so one is led to believe that they are intelligent—not looking at the hidden failures. As to academic science: a few years ago, the great Anglo-Lebanese mathematician Michael Atiyah of string theory fame came to New York to raise funds for a research center in mathematics based in Lebanon. In his speech, he enumerated applications in which mathematics turned out to be useful for society and modern life, such as traffic signaling. Fine. But what about areas where mathematics led us to disaster (as in, say, economics or finance, where it blew up the system)? And how about areas out of the reach of mathematics? I thought right there of a different project: a catalog of where mathematics fails to produce results, hence causes harm.

Cherry-picking has optionality: the one telling the story (and publishing it) has the advantage of being able to show the confirmatory examples and completely ignore the rest—and the more volatility and dispersion, the rosier the best story will be (and the darker the worst story). Someone with optionality—the right to pick and choose his story—is only reporting on what suits his purpose. You take the upside of your story and hide the downside, so only the sensational seems to count.

The real world relies on the intelligence of antifragility, but no university would swallow that—just as interventionists don’t accept that things can improve without their intervention. Let us return to the idea that universities generate wealth and the growth of useful knowledge in society. There is a causal illusion here; time to bust it.

1 Is democracy epiphenomenal? Supposedly, democracy works because of this hallowed rational decision making on the part of voters. But consider that democracy may be something completely accidental to something else, the side effect of people liking to cast ballots for completely obscure reasons, just as people enjoy expressing themselves just to express themselves. (I once put this question at a political science conference and got absolutely nothing beyond blank nerdy faces, not even a smile.)