Clinical versus Statistical Prediction: Sources of Resistance
Despite robust evidence that actuarial (statistical) methods outperform clinical judgment in prediction tasks, clinicians often resist using them. This resistance stems from unfamiliarity with the research, misconceptions about group statistics, overconfidence in subjective judgment, and professional values that prioritize intuition.
THE IDEA
LACK OF IMPACT AND SOURCES OF RESISTANCE
The review describes limited uptake of research comparing clinical and statistical judgment, particularly in clinical psychology. Guilmette et al.’s (1990) survey found that most psychologists specializing in brain damage assessment preferred procedures lacking actuarial methods over those with formulas available. The authors describe interviews as central to training and licensing at the time, yet report little use of actuarial interpretation despite evidence of its predictive value.
Lack of impact is sometimes due to lack of familiarity with the scientific evidence. Some clinicians are unaware of the comparative research and do not even realize an issue exists. Others still refer to earlier studies and claim that the clinician was handicapped, unaware of the subsequent research that has rendered these arguments counterfactual.
Others who know the evidence may still dismiss it based on tendentiousness or misconception. Mental health professionals’ education, training, theoretical orientations and identifications, and personal values may dictate against recognition of the actuarial advantage. Some psychologists, for example, believe that the use of a predictive equation dehumanizes their clients. The position overlooks the human costs of increased error that may result.
A common anti-actuarial argument, or misconception, is that group statistics do not apply to single individuals or events. The argument abuses basic principles of probability. Although individuals and events may exhibit unique features, they typically share common features with other persons or events that permit tallied observations or generalizations to achieve predictive power. An advocate of this anti-actuarial position would have to maintain, for the sake of logical consistency, that if one is forced to play Russian roulette a single time and is allowed to select a gun with one or five bullets in the chamber, the uniqueness of the event makes the choice arbitrary.
Finally, subjective appraisal may lead to inflated confidence in the accuracy of clinical judgment and the false impression that the actuarial method is inferior. Derivation and cross-validation of an actuarial method yields objective information on how well it does and does not perform (Dawes, 1979). When the clinician reviews research that shows, for example, that the Goldberg Rule for the MMPI achieved 70% accuracy in a comparable setting and exceeded the performance of 29 judges in the study, this may still seem to compare unfavorably to self-perceived judgmental powers. The immediacy and salience of clinical experience fosters the misappraisal. The clinician may recall dramatic instances in which his interpretations proved correct or in which he avoided error by countervailing an actuarial conclusion, failing to recognize or correctly tally counter-instances.
Ultimately, then, clinicians must choose between their own observations or impressions and the scientific evidence on the relative efficacy of the clinical and actuarial methods. The factors that create difficulty in self-appraisal of judgmental accuracy are exactly those that scientific procedures, such as unbiased sampling, experimental manipulation of variables, and blind assessment of outcome, are designed to counter. Failure to accept a large and consistent body of scientific evidence over unvalidated personal observation may be described as a normal human failing or, in the case of professionals who identify themselves as scientific, plainly irrational.
When to use it
A hospital compares a readmission model’s predictions with clinicians’ predictions and overrides, recording outcomes to assess which improves accuracy.
An HR department compares unstructured interviews with a method that combines structured interview scores using a specified prediction rule.
A parole board relies on a statistical risk assessment tool to decide release, but board members sometimes disregard it due to subjective impressions of the offender.
In psychological assessment, clinicians prefer projective tests (e.g., Rorschach) over actuarial measures like the MMPI, despite evidence of lower predictive validity.
FROM THEORY TO EVERYDAY LIFE
In practice
MMPI Goldberg Rule Study
The Goldberg Rule for the MMPI achieved 70% accuracy in a comparable setting and exceeded the performance of 29 judges in the study. Despite this evidence, clinicians often perceive their own judgment as superior.
Russian Roulette Analogy
A common anti-actuarial argument claims that group statistics do not apply to single individuals. The text refutes this by analogy: if forced to play Russian roulette once and allowed to choose a gun with one or five bullets, the uniqueness of the event does not make the choice arbitrary.
A MOMENT TO REFLECT
Put it to work.
Think of a time when you relied on intuition rather than statistics or data to make a decision. What factors influenced that choice? How might an actuarial approach have changed the outcome?
Why do professionals often resist using statistical prediction tools even when evidence shows their superiority? What cognitive biases might be at play?
Consider a domain like medical diagnosis or employee selection. How could you design a simple check to compare the accuracy of your clinical judgment versus a statistical rule?
Often confused with
References
- Guilmette et al. Survey on Brain Damage Assessment — Guilmette, T. J., et al. (1990)
- Dawes on Validation of Actuarial Methods — Dawes, R. M. (1979)
CONTINUE THE CONVERSATION
Source texts
- Heuristics and Biases The Psychology of Intuitive Judgment (Thomas Gilovich, Dale Griffin, Daniel Kahneman)