The Myth of the Myth of the Hot Hand

The Myth of the Myth of the Hot Hand

We know B. F. Skinner as a psychologist, in many ways the modern psychologist, the man who stared down the Freudians and led a competing psychology, behaviorism, concerned only with what was visible and what could be measured, requiring no hypotheses about unconscious or, for that matter, conscious motivations. For Skinner, a theory of mind just was a theory of behavior, and the interesting projects for psychologists thus did not concern thoughts or feelings at all, but rather the manipulation of behavior by means of reinforcement.

Less well known is Skinner’s history as a frustrated novelist. Skinner was an English major at Hamilton College and spent much of his time with Percy Saunders, a chemistry professor and aesthete whose house was a kind of literary salon. Skinner read Ezra Pound, and listened to Schubert, and wrote adolescently heated poems (“ At night, he stops, breathless / Murmuring to his earthly consort / ‘Love exhausts me!’”) for the college literary magazine. He did not take a single psychology course. After college, Skinner attended the Bread Loaf writer’s conference, where he wrote “ a one-act play about a quack who changed people’s personalities with endocrines” and succeeded in pressing several of his short stories on Robert Frost. Frost wrote Skinner a very satisfactory letter praising his stories and counseling: “ All that makes a writer is the ability to write strongly and directly from some unaccountable and almost invincible personal prejudice. . . . I take it that everybody has the prejudice and spends some time feeling for it to speak and write from. But most people end as they begin by acting out the prejudices of other people.”

Thus encouraged, Skinner moved into his parents’ attic in Scranton in the summer of 1926 and set out to write. But Skinner found it was not so easy to find his own personal prejudice, or, having found it, to put it in literary form. His time in Scranton came to nothing; he managed a couple of stories and a sonnet about labor leader John Mitchell, but spent his time mostly building model ships and tuning in to distant signals from Pittsburgh and New York on the radio, then a brand-new procrastination device.

“ A violent reaction against all things literary was setting in,” he later wrote of this period. “I had failed as a writer because I had nothing important to say, but I could not accept that explanation. It was literature which must be at fault.” Or, more bluntly: “ Literature must be demolished.”

Skinner was a regular reader of the literary magazine The Dial; in its pages, he encountered the philosophical writings of Bertrand Russell, and via Russell was brought to John Watson, the first great advocate of the behaviorist outlook that would soon become almost synonymous with Skinner’s name. Watson held that scientists were in the business of observing the results of experiments, and only that; there was no room for hypotheses about consciousness or souls. “ No one has ever touched a soul or seen one in a test-tube,” he famously wrote, by way of dismissing the notion. These uncompromising words must have thrilled Skinner, as he moved to Harvard as a graduate student in psychology, making ready to banish the vague, unruly self from the scientific study of behavior.

Skinner had been much struck by an experience of spontaneous verbal production he’d experienced in his lab; a machine in the background was making a repetitive, rhythmic sound, and Skinner found himself talking along with it, following the beat, silently repeating the phrase “ You’ll never get out, you’ll never get out, you’ll never get out.” What seemed like speech, or even, in a small way, like poetry, was actually the result of a kind of autonomous verbal process, requiring nothing like a conscious author. This provided just the idea Skinner needed to settle his score with literature. What if language, even the language of the great poets, was just another behavior, trained by exposure to stimuli, and manipulable in the lab?

In college, Skinner had written imitations of Shakespeare’s sonnets; he retrospectively described this experience, in thoroughly behaviorist fashion, as “the strange excitement of emitting whole lines ready-made, properly scanned and rhymed.” Now, as a young psychology professor in Minnesota, he recast Shakespeare himself as more emitter than writer. This approach was not as crazy then as it seems now; the dominant form of literary criticism at the time, “close reading,” bore the mark of Watson’s philosophy (though not as directly as Skinner did), displaying a very behaviorist preference for the words on the page over the unobservable intentions of the author.

Shakespeare is famous as a master of the alliterative line, in which several words in close succession start with the same sound (“Full fathom five thy father lies . . .”). For Skinner, this argument by example was no kind of science. Did Shakespeare alliterate? If he did, then math could prove it so. “ Proof that there is a process responsible for alliterative patterning,” he wrote, “can be obtained only through a statistical analysis of all the arrangements of initial consonants in a reasonably large sample.” And what form of statistical analysis? None other than a form of Fisher’s p-value test. Here, the null hypothesis is that Shakespeare paid no heed to the initial sounds of words at all, so that the first letter of one word of poetry has no effect on other words in the same line. The protocol was much like that of a clinical trial, but with one big difference: the biomedical researcher testing a drug hopes with all his heart to see the null hypothesis refuted, and the effectiveness of the medicine demonstrated. For Skinner, aiming to knock literary criticism off its plinth, the null hypothesis was the attractive one.

Under the null hypothesis, the frequency with which initial sounds appeared multiple times in the same line would be unchanged if the words were put in a sack, shaken up, and laid out again in random order. And this is just what Skinner found in his sample of a hundred sonnets. Shakespeare failed the significance test. Skinner writes:

“In spite of the seeming richness of alliteration in the sonnets, there is no significant evidence of a process of alliteration in the behavior of the poet to which any serious attention should be given. So far as this aspect of poetry is concerned, Shakespeare might as well have drawn his words out of a hat.”

“Seeming richness”—what chutzpah! It captures perfectly the spirit of the psychology that Skinner wanted to create. Where Freud had claimed to see what had previously been hidden, repressed, or obscured, Skinner wanted to do the opposite—to deny the existence of what seemed in plain view.

But Skinner was wrong; he hadn’t proved that Shakespeare didn’t alliterate. A significance test is an instrument, like a telescope. And some instruments are more powerful than others. If you look at Mars with a research-grade telescope, you’ll see moons; if you look with binoculars, you won’t. But the moons are still there! And Shakespeare’s alliteration is still there. As documented by literary historians, it was a standard device of the time, known to and consciously deployed by nearly everyone writing in English.

Skinner’s test had not detected a statistically significant surplus of repeated sounds under its model. That is a narrower finding than proving there was no deliberate alliteration. But why would it? The use of alliteration in poetry is both positive and negative; in certain places you alliterate to create an effect, and in other places you intentionally avoid it, lest you create an effect you don’t want. It may be that the overall tendency is to increase the number of alliterative lines, but even if so, the increase should be small. Stuff your sonnets with one or two extra alliterations each and you become one of the stone-footed poets mocked by Shakespeare’s fellow Elizabethan George Gascoigne: “ Many writers indulge in repeticion of sundrie wordes all beginning with one letter, the whiche (beyng modestly used) lendeth good grace to a verse; but they do so hunt a letter to death, that they make it Crambe, and Crambe bis positum mors est.” The Latin phrase means “Cabbage served twice is death.” Shakespeare’s writing is rich in effect, but always restrained. He would never pack in so much cabbage that Skinner’s crude test could smell it.

A statistical study that’s not refined enough to detect a phenomenon of the expected size is called underpowered—the equivalent of looking at the planets with binoculars. Moons or no moons, you get the same result, so you might as well not have bothered. You don’t send binoculars to do a telescope’s job. Low power creates a different problem from mistaking statistical significance for practical importance. A high-powered study can detect a small effect whose practical consequences still need to be assessed. An underpowered one may lead you to wrongly dismiss a small effect that your method was simply too weak to see.

Consider Spike Albrecht. The freshman guard for Michigan’s men’s basketball team, standing at just five foot eleven and a bench player most of the season, wasn’t expected to play a big role when the Wolverines faced Louisville in the 2013 NCAA final. But Albrecht made five straight shots, four of them three-pointers, in a ten-minute span in the first half, leading Michigan to a ten-point lead over the heavily favored Cardinals. He had what basketball fans call “the hot hand”—the apparent inability to miss a shot, no matter how great the distance or how fierce the defense.

Except there’s supposed to be no such thing. In 1985, in one of the most famous contemporary papers in cognitive psychology, Thomas Gilovich, Robert Vallone, and Amos Tversky (hereafter GVT) did to basketball fans what B. F. Skinner had done to lovers of the Bard. They obtained records of every shot taken by the 1980−81 Philadelphia 76ers in their forty-eight home games and analyzed them statistically. If players tended toward hot streaks and cold streaks, you might expect a player to be more likely to hit a shot following a basket than a shot following a miss. And when GVT surveyed NBA fans, they found this theory had broad support; nine out of ten fans agreed that a player is more likely to sink a shot when he’s just hit two or three baskets in a row.

But nothing of the kind was going on in Philadelphia. Julius Erving, the great Dr. J, was a 52% shooter overall. After three straight baskets, a situation that you’d think might indicate Erving was hot, his percentage went down to 48%. And after three straight misses, his field goal percentage didn’t drop, but rather stayed right at 52%. For other players, like Darryl “Chocolate Thunder” Dawkins, the effect was even more extreme. After a hit, his overall 62% shooting percentage dipped to 57%; after a miss, it shot up to 73%, exactly the opposite of the fan predictions. (One possible explanation: a missed shot suggests Dawkins was facing effective defenders on the perimeter, inducing him to drive to the basket for one of his trademark backboard-shattering dunks, which he gave names like “In Your Face Disgrace” and “Turbo Sexophonic Delight.”)

Does this mean there’s no such thing as the hot hand? Not just yet. A brief change in performance need not produce an overall tendency for hits to follow hits across an entire season. Spike Albrecht’s ten-minute stretch poses a question about short intervals. GVT also examined sequences of four shots. To assess what such a test can rule out, we need to ask which sizes and durations of change it could reliably detect. That is the statistical-power question connecting the basketball analysis with Skinner’s test of Shakespeare.