Discrimination (in probability judgment)

Discrimination

Discrimination is the extent to which – irrespective of their numeric labels, which are central to the calibration notion – the judgments in a given collection sharply distinguish instances in which the target event occurs from those in which it does not (Yates, 1990, 1994, 1998). Roughly, discrimination corresponds to d or the area under an ROC curve in ROC or signal detection theory analyses (cf. Swets, 1986). Consistent with Eq. 2, overall accuracy is a compensatory function entailing both calibration and discrimination, as made precise in Murphy's (1973) decomposition of (see also Yates, 1982). Thus, a weakness in calibration can be offset by a strength in discrimination, and vice versa. There is a compelling case, however, that one should prefer good discrimination over good calibration when given the choice. For example, whereas good calibration often can be achieved by simple mathematical transformations (e.g., adding a constant to every probability judgment), good discrimination demands access to solid, predictive evidence and skill at exploiting that evidence, which are difficult to find in any real-life, practical situation. Such arguments make cross-cultural comparisons especially interesting. In particular, in the Yates et al. (1989) study, the weak calibration of the Chinese participants' judgments coupled with their overall accuracy being equivalent to that of the Americans and Japanese suggested that their discrimination might be especially good. If this were so, it might suggest that elements of Chinese culture are somehow able to foster judgment customs that facilitate this most appealing strength. Yates et al. (1989) did indeed observe that the discrimination of their Chinese respondents' general knowledge judgments was superior to that of their American and Japanese participants, even though the comparison with the latter group was not statistically significant.

Interpreting the strong Chinese discrimination in the Yates et al. (1989) study is complicated by the fuzzy meaning of the discrimination concept for judgments about general knowledge (see Yates, 1982, for an elaboration of the difficulties). This ambiguity was a primary reason Yates et al. (1998) undertook their artificial medical ecology study involving the fictitious diseases Trebitis and Philiosis. They were able to define the target event as A = "The patient has Trebitis (rather than Philiosis)." The meaning of discrimination is completely straightforward when target events are defined externally this way rather than internally as in general knowledge studies (i.e., "My chosen answer is correct"). This permitted a cleaner test of the possibility of especially strong Chinese discrimination skills. However, such strength did not emerge. Recall that, in the first study by Yates et al. (1998), the computer automatically provided the participant as "physician" with all the available symptoms for every patient. In that study, the discrimination of the Chinese participants' judgments was significantly weaker, not stronger, than that of the American and Japanese participants. Also recall that in the second study, the participants themselves determined which symptoms they saw for each patient. Importantly, this discretion wiped out all differences in discrimination among the participant groups, a result to which we will return.