Scale Compatibility Effect

Study 1: Prediction of Market Value

In our first study, 77 Stanford students were presented with a list of 12 well-known U.S. companies taken from the 1987 Business Week Top 100. For each company, students were given two items of information: 1986 market value (the total value of the outstanding shares in billions of dollars) and 1987 profit standing (rank of the company in terms of its 1987 earnings among the Top 100). Half the subjects were asked to predict 1987 market value (in billions of dollars). They were informed that, among the Top 100, the highest market value in 1987 was $68.2 billion and that the lowest was $5.1 billion, so their predictions should fall within that range. The remaining subjects were asked to predict each company’s rank (from 1 to 100) in market value for 1987. Thus, both groups of subjects received identical information and predicted the same criterion using a different response scale. Although the two response scales differ in units (dollar versus rank) and direction (low rank means high market value), the two dependent variables should yield the same ordering of the 12 companies. To encourage careful consideration, a $75 prize was offered for the person whose predictions most nearly matched the actual values.

The compatibility hypothesis states that a predictor is weighted more heavily when it matches the response scale than when it does not. That is, 1986 market value in dollars (D) should be weighted more heavily by the subjects who predict in dollars (d) than by those who predict in rank (r). By the same token, 1987 profit rank (R) should be weighted more heavily by the subjects who predict in rank than by those who predict in dollars. To investigate this hypothesis, we correlated the criteria with the predictors, estimated the relative weights of the two predictors, and devised a statistical test based on reversals of order.

The product-moment correlations of d with D and R were .93 and .77, respectively, whereas the correlations of r with D and R were .74 and .94, respectively. Thus, the correlation between the matched variables was higher than that between the nonmatched variables. It is instructive to examine the compatibility effect in terms of the relative weights of the two predictors in a multiple-regression equation. These values can be computed directly or derived from the correlations between the predictors and the criterion together with the correlation between the predictors. (To make the regression weights positive, the ranking order was reversed.) The multiple regressions for both dollars and ranks fit the average data very well with multiple correlations of .99. Let d and r denote the mean observed predictions of 1987 dollar value and rank, respectively, for a company whose 1986 dollar value is Dᵢ and whose 1987 profit rank is Rᵢ. The multiple-regression equations, then, take the form

dᵢ = α_d Dᵢ + β_d Rᵢ

rᵢ = α_r Dᵢ + β_r Rᵢ,

when the independent variables are expressed in standardized units. Thus, α_d and α_r are the regression weights for the 1986 market value (Dᵢ) estimate from the predicted dollars and ranks, respectively. Similarly, β_d and β_r are the corresponding weights for the second predictor, 1987 profit rank. The relative weights for the first predictor in each of the two response modes are

A_d = α_d / (α_d + β_d)

A_r = α_r / (α_r + β_r).

These values measure the relative contribution of Dᵢ in the prediction of dollars and rank, respectively. If the weighting of the dimensions is independent of the response scale, A_d and A_r are expected to be equal, except for minor perturbations due to a nonlinear relation between d and r. As we argue next, the compatibility hypothesis implies A_d > A_r. Note that A_d is the relative weight of the 1986 market value in dollars, estimated from the prediction of dollars, whereas A_r is the relative weight of the same variable estimated from the prediction of rank. The first index reflects the impact of Dᵢ in a compatible condition (i.e., when the predictions are made in dollars), whereas A_r reflects the impact of Dᵢ in the less compatible condition (i.e., when the predictions are made in ranks). If the compatibility between the predictor and the criterion enhances the weight of that variable, then A_d should exceed A_r.

The values estimated from the regression equations were A_d = .64 and A_r = .32, in accord with the compatibility hypothesis. Thus, Dᵢ was weighted more than Rᵢ in the prediction of dollars, whereas Rᵢ was weighted more than Dᵢ in the prediction of rank. Moreover, each predictor was weighted about twice as much in the compatible condition as it was in the noncompatible condition. When interpreting the relative weights, here and in later studies, we should keep in mind that they are based on aggregate data, that the predictors (D and R) are correlated, and that the relation between the two criteria (d and r) should be monotone but not necessarily linear. Although these factors do not account for the discrepancy between A_d and A_r, it is desirable to obtain a purely ordinal test of the compatibility hypothesis within the data of each subject that is not open to these objections. The following analysis of order reversals provides a basis for such a test.

The change in the relative weights induced by the response mode could produce reversals in the order of the predictions. In the present study, there were 21 pairs of companies (i, j) in which Dᵢ > Dⱼ and Rⱼ > Rᵢ. If D is weighted more heavily than R in the subject’s prediction of dollars and R is weighted more heavily than D in the subject’s prediction of rank, we would expect dᵢ > dⱼ and rⱼ > rᵢ. The data confirmed this hypothesis. Subjects who predicted dollars favored the company with the higher D 72% of the time, whereas subjects who predicted rank favored the company with the higher D only 39% of the time. (Ties were excluded from this analysis.) This difference is highly significant (p < .001). Note that the subjects did not directly compare the companies; the ordering was inferred from their predictions.