Coherent Arbitrariness

Figure 13: Components of judgment variance

Outrage and punitive intent were both measured on similar rating scales, defined more or less clearly by verbal labels. The dollar scale belongs to a different family, which is far more problematic.

Dollars and Anchors

The title of our academic paper expresses its central message: “Shared Outrage and Erratic Awards: The Psychology of Punitive Damages.” There was a fair amount of agreement among our experimental jurors in their ratings of punitive intent; the ratings were mostly explained by outrage. However, the dollar measure most closely simulated the courtroom situation, and it was unacceptably noisy.

The reason is not mysterious. A person asked to assign punitive damages in a case may feel that the chosen dollar amount is essentially arbitrary. The feeling of arbitrariness conveys important information: it tells you that other people will make widely different arbitrary decisions and that the judgments will be very noisy. This turns out to be a characteristic of the family of scales to which dollar awards belong.

The legendary Harvard psychologist S. S. Stevens discovered the surprising fact that people share strong intuitions about the ratios of intensity of many subjective experiences and attitudes. They can adjust a light so that it appears “twice as bright” as another, and they agree that the emotional significance of a ten-month prison sentence is not nearly ten times as bad as that of a sentence of one month. Stevens called scales that draw on such intuitions ratio scales.

You can tell that our intuitions about money are expressed in ratios from the ease with which we understand such expressions as “Sara got a 60% raise!” or “Our rich neighbor lost half his wealth overnight.” The dollar scale of punitive damages is a ratio scale for the measurement of the intention to punish. It has a meaningful zero (zero dollars), and in this task it has no specified upper bound.

Stevens discovered that a ratio scale (like the dollar scale) can be tied down by a single intermediate anchor (the jargon term is modulus). In his laboratory, he would expose observers to a light of a certain brightness, with the instruction to “call the brightness of that light 10 (or 50, or 200) and assign numbers to other brightnesses accordingly.” As expected, the numbers that observers assigned to lights of different brightness were proportional to the arbitrary anchor they were instructed to adopt. An observer who was anchored on the number 200 would make judgments that were 20 times higher than if the anchor had been 10; the standard deviation of the observer’s judgments would also be proportional to the anchor.

The source book’s chapter 13 describes an anchoring experiment, in which people’s willingness to pay for an object was strongly influenced by asking them first if they would pay (in dollars) the last two digits of their Social Security number. A more striking result was that the initial anchor also affected their willingness to pay for a whole list of other objects. Participants who were induced to agree to pay a large amount for a cordless trackball agreed to pay a correspondingly larger amount for a cordless keyboard. It appears that people are much more sensitive to the relative value of comparable goods than to their absolute value. The authors of the study named the persistent effect of a single anchor “coherent arbitrariness.”

Consider a hypothetical anchor for a punitive-damages case involving severe harm to Joan Glover. Suppose the following information were supplied:

In a similar case involving another pharmaceutical company, a little girl who was the victim suffered moderate psychological trauma (as in the mild-harm version you read earlier). The punitive damages were set at $1.5 million.

Notice that the problem of setting a punishment for General Assistance has suddenly become much easier. Indeed, an amount may have come to your mind already. There is a multiplier (or ratio) of the dollar awards that corresponds to the contrast between the severe harm that was done to Joan and the mild harm suffered by the other little girl. Furthermore, the single anchor you read ($1.5 million) is sufficient to tie down the entire dollar scale of punishment. It is now easy for you to set damages for cases both more severe and milder than the two considered so far.

If anchors are required to make judgments on a ratio scale, what happens when people are not given an anchor? Stevens reported the answer. In the absence of guidance from the experimenter, people are forced to make an arbitrary choice when they use the scale for the first time. From that point, they make their judgments consistently, using their first answer as an anchor.

Participants in the punitive-damages study faced a task without a supplied anchor. Like the anchorless observers in Stevens’s lab, they had to choose an initial dollar level for the first case. They then set punitive damages for nine other cases. These nine judgments were not arbitrary because they could be made consistent with the initial anchoring judgment, and therefore with one another.

The findings of Stevens’s laboratory suggest that the anchor that individuals produce should have a large effect on the absolute values of their subsequent dollar judgments but no effect whatsoever on the relative positions of the ten cases. A large initial judgment causes all the other judgments to be proportionately large without affecting their relative size. This reasoning leads to a surprising conclusion: although they appear hopelessly noisy, dollar judgments actually reflect the judges’ punitive intentions. To discover these intentions, we need only replace the absolute dollar values with relative scores.

To test this idea, we repeated the analysis of noise after replacing each dollar award by its rank among an individual’s ten judgments. The highest dollar award was scored 1, the next highest was scored 2, and so forth. This transformation removes differences in jurors’ overall dollar levels, because the 1 to 10 distribution of ranks is the same for everyone, except for occasional ties. (In case you wondered, there were multiple versions of the questionnaire because each individual judged ten of the twenty-eight scenarios. We conducted the analysis separately for each group of participants who had responded to the same ten scenarios, and we report an average.)

The results were striking: the proportion of noise in the judgments dropped from 94% to 49% (figure 14). Transforming the dollar awards into rankings revealed that jurors were actually in substantial agreement about the appropriate punishment in different cases. Indeed, the rankings of dollar awards were, if anything, slightly less noisy than the original ratings of punitive intent.