Berkson's Fallacy (Berkson's Paradox)

BERKSON'S FALLACY, OR: WHY ARE HANDSOME MEN SUCH JERKS?

That correlations can arise from unseen common causes is confusing enough, but that's not the end of the story. Correlations can also come from common effects. This phenomenon is known as Berkson's fallacy, after the medical statistician Joseph Berkson.

Berkson himself was, like the statistician Ronald Fisher, a vigorous skeptic about the link between tobacco and cancer. Berkson, an MD, represented the old school of epidemiology, deeply suspicious of any claim whose support was more statistical than medical. Such claims, he felt, represented a trespass by naive theorists onto ground that rightfully belonged to the medical profession. "Cancer is a biologic, not a statistical problem," he wrote in 1958. "Statistics can soundly play an ancillary role in its elucidation. But if biologists permit statisticians to become arbiters of biologic questions, scientific disaster is inevitable."

Berkson was especially troubled by the fact that tobacco use was found to be correlated not only with lung cancer but with dozens of other diseases, afflicting every system of the human body. For Berkson, the idea that tobacco could be so thoroughgoingly poisonous was inherently implausible: "It is as though, in investigating a drug that previously had been indicated to relieve the common cold, the drug was found not only to ameliorate coryza, but to cure pneumonia, cancer, and many other diseases. A scientist would say, 'There must be something wrong with this method of investigation.'"

Berkson, like Fisher, was more apt to believe the "constitutional hypothesis," that some preexisting difference between nonsmokers and smokers accounted for the relative healthiness of the abstainers:

If 85 to 95 per cent of a population are smokers, then the small minority who are not smokers would appear, on the face of it, to be of some special type of constitution. It is not implausible that they should be on the average relatively longevous, and this implies that death rates generally in this segment of the population will be relatively low. After all, the small group of persons who successfully resist the incessantly applied blandishments and reflex conditioning of the cigaret advertisers are a hardy lot, and, if they can withstand these assaults, they should have relatively little difficulty in fending off tuberculosis or even cancer!

Berkson also objected to the original study of Doll and Hill, which was conducted among patients in British hospitals. What Berkson had observed in 1938 was that selecting patients in this way can create the appearance of associations that aren't really there.

Suppose, for example, you want to know whether high blood pressure is a risk factor for diabetes. You might take a survey of the patients in your hospital, with the goal of finding out whether high blood pressure was more common among the nondiabetics or the diabetics. And you find, to your surprise, that high blood pressure is less common among the patients with diabetes. You might thus be inclined to conclude that high blood pressure was protective against diabetes, or at least against diabetic symptoms so severe as to require hospitalization. But before you start advising diabetic patients to ramp up their consumption of salty snacks, consider this table:

1,000 total population

300 people with high blood pressure

400 people with diabetes

120 people with both high blood pressure and diabetes

Suppose there are a thousand people in our town, of whom 30% have high blood pressure and 40% have diabetes. (We like salty snacks and sweet snacks in our town.) And let's suppose, furthermore, that there's no relation between the two conditions; so 30% of the 400 diabetics, or 120 people in all, suffer from high blood pressure as well.

If all the sick people in town wind up in the hospital, then your hospital population is going to consist of:

• 180 people with high blood pressure but no diabetes
• 280 people with diabetes but no high blood pressure
• 120 people with both high blood pressure and diabetes.

Of the 400 total diabetics in the hospital, 120, or 30%, have high blood pressure. But of the 180 nondiabetics in the hospital, 100% have high blood pressure! It would be nuts to conclude from this that high blood pressure keeps you from having diabetes. The two conditions are negatively correlated, but that's not because one causes the absence of the other. It's also not because there's a hidden factor that both raises your blood pressure and helps regulate your insulin. It's because the two conditions have a common effect—namely, they put you in the hospital.

To put it in words: if you're in the hospital, you're there for a reason. If you're not diabetic, that makes it more likely the reason is high blood pressure. So what looks at first like a causal relationship between high blood pressure and diabetes is really just a statistical phantom.

The effect can work the other way, too. In real life, having two diseases is more likely to land you in the hospital than having one. Maybe the 120 hypertensive diabetics in our town all end up in the hospital, but 90% of the relatively healthy folks with only one thing wrong with them stay home. What's more, there are other reasons to be in the hospital; for instance, on the first snowy day of the year, a lot of people try to clean out their snowblower with their hand and get their finger chopped off. So the hospital population might look like:

• 10 people with no diabetes or high blood pressure but a severed finger
• 18 people with high blood pressure but no diabetes
• 28 people with diabetes but no high blood pressure
• 120 people with both high blood pressure and diabetes

Now, when you do your hospital study, you find that 120 out of 148 diabetics, or 81%, have high blood pressure. But only 18 of the 28 nondiabetics, or 64%, have high blood pressure. That makes it seem that high blood pressure makes you more likely to have diabetes. But again, it's an illusion; all we're measuring is the fact that the set of people who end up in the hospital is anything but a random sample of the population.

Berkson's fallacy makes sense outside the medical domain; in fact, it even makes sense outside the realm of features that can be precisely quantified. You may have noticed that, among the men in your dating pool, the handsome ones tend not to be nice, and the nice ones tend not to be handsome. Is that because having a symmetrical face makes you cruel? Or because being nice to people makes you ugly? Well, it could be. But it doesn't have to be. I present below the Great Square of Men:

and I take as a working hypothesis that men are in fact equidistributed all over this square; in particular, there are nice handsome ones, nice ugly ones, mean handsome ones, and mean ugly ones, in roughly equal numbers.

But niceness and handsomeness have a common effect; they put these men in the group that you notice. Be honest—the mean uglies are the ones you never even consider. So inside the Great Square is a Smaller Triangle of Acceptable Men:

And now the source of the phenomenon is clear. The handsomest men in your triangle run the gamut of personalities, from kindest to cruelest. On average, they're about as nice as the average person in the whole population, which, let's face it, is not that nice. And by the same token, the nicest men are only averagely handsome. The ugly guys you like, though—they make up a tiny corner of the triangle, and they are pretty darn nice—they have to be, or they wouldn't be visible to you at all. The negative correlation between looks and personality in your dating pool is absolutely real. But if you try to improve your boyfriend's complexion by training him to act mean, you've fallen victim to Berkson's fallacy.

Literary snobbery works the same way. You know how popular novels are terrible? It's not because the masses don't appreciate quality. It's because there's a Great Square of Novels, and the only novels you ever hear about are the ones in the Acceptable Triangle, which are either popular or good. If you force yourself to read unpopular novels chosen essentially at random—I've been on a literary prize jury, so I've actually done this—you find that most of them, just like the popular ones, are pretty bad.

The Great Square is too simple by far, of course. There are many dimensions, not just two, along which you can rate your love interests or your weekly reading. So the Great Square is better described as a kind of Great Hypercube. And that's just for your own personal preferences! If you try to understand what happens in the whole population, you need to grapple with the fact that different people define attractiveness differently; they may differ about what weights to place on various criteria, or they may simply have incompatible preferences. The process of aggregating opinions, preferences, and desires from many different people presents yet another set of difficulties. Understanding those differences requires more than the simple selection model.