Clinical vs. Actuarial Judgment

Clinical and Actuarial Judgment

Clinical judgment combines information through a person’s assessment; actuarial judgment uses an explicit statistical or mechanical rule. The distinction concerns how information is combined, not whether the person works in medicine. The literature reviewed here compares the accuracy of the two methods on the same prediction or classification tasks.

Across the reviewed research, a consistent pattern emerged. Eliminating research that did not protect sufficiently against inflated results for actuarial methods, there remain nearly 100 comparative studies in the social sciences. In virtually every one of these studies, the actuarial method has equaled or surpassed the clinical method, sometimes slightly and sometimes substantially (Meehl, 1986; Sawyer, 1966). For example, in Watley and Vance’s (1974) study on the prediction of college grades the methods tied; in Carroll et al.’s (1982) study on the prediction of parole violation, the actuarial method showed a slight to modest advantage; and in Wittman’s (1941) study on the prediction of response to electroshock therapy, the actuarial method was correct almost twice as often as the clinical method.

The earlier comparative studies were often met with doubts about validity and generalization. It was claimed, for example, that the studies misrepresented the clinical method either by denying judges access to crucial data sources such as interviews, by using artificial tasks that failed to tap their areas of expertise, or by including clinicians of questionable experience or expertise.

The evidence that has accumulated over the years meets these challenges. First, numerous studies have examined judgments that are not artificial but common to everyday practice and for which special expertise is claimed. Studies discussed by the source involved distinguishing less serious from major psychiatric disorder, detecting brain damage, and predicting survival time. Other studies have examined the diagnosis of medical versus psychiatric disorder (Oskamp, 1962); the description or characterization of personality (Halbower, 1955); and the prediction of treatment outcome (Barron, 1953), length of psychiatric hospitalization (Dunham & Meltzer, 1946), and violent behavior (Werner, Rose, & Yesavage, 1983). These are decisions that general practitioners or specialists often address, and in a number of studies investigators did not introduce judgment tasks that clinicians then performed, but rather examined decisions already made in the course of everyday practice.

Other studies have provided clinicians or judges with access to preferred sources of information. Even in 1966, Sawyer was able to locate 17 comparisons between actuarial and clinical judgment based on the results of psychological testing and interview. Other investigators have allowed judges to collect whatever data they preferred in whatever manner they preferred. In Carroll et al.’s (1982) naturalistic study on the prediction of parolees’ behavior after release, the parole board did not alter the data collection procedures. In Dawes’s (1971) study on the prediction of graduate student performance, the admissions committee relied on the same data normally used to reach decisions. None of the 17 comparisons reviewed by Sawyer and neither of the studies by Carroll et al. or Dawes favored clinical over actuarial judgment.

Nor has the outcome varied within or across studies involving judges at various levels of experience or expertise. In Goldberg’s (1965) study, novice and experienced MMPI interpreters performed similarly when using the clinical method and neither group surpassed the actuarial method, results parallel to those of Leli and Filskov (1984) in their study on the detection of brain damage. Other studies on the detection and localization of brain damage have yielded similar results (Leli & Filskov, 1981; Wedding, 1983). For example, Wedding found that neither clinicians with extensive experience interpreting the tests under study nor a nationally prominent neuropsychologist surpassed the overall accuracy of actuarial methods in determining the presence, location, and cause of brain damage.

The comparative studies often do not permit general conclusions about the superiority of one or another specific actuarial decision rule. Some studies, such as Goldberg’s, do show application across settings, but much of the research has involved restricted samples. Investigators have been less interested in a specific procedure’s range of application than in performing an additional test of the two methods and thereby extending the range of comparative studies.

The various studies can thus be viewed as repeated sampling from a universe of judgment tasks involving the diagnosis and prediction of human behavior. Lacking complete knowledge of the elements that constitute this universe, representativeness cannot be determined precisely. However, with a sample of about 100 studies and the same outcome obtained in almost every case, it is reasonable to conclude that the actuarial advantage is not exceptional but general and likely encompasses many of the unstudied judgment tasks. Stated differently, if one poses the query: “Would an actuarial procedure developed for a particular judgment task (say, predicting academic success at my institution) equal or exceed the clinical method?” The available research places the odds solidly in favor of an affirmative reply. “There is no controversy in social science that shows such a large body of qualitatively diverse studies coming out uniformly . . . as this one” (Meehl, 1986, p. 373).