Heuristic Elicitation Design
The heuristic elicitation design tests the substitution hypothesis by having one group judge a target attribute and another group judge a hypothesized heuristic attribute, then comparing the judgments in comparable units.
THE IDEA
IDENTIFYING A HEURISTIC
Hypotheses about judgment heuristics have most often been studied by examining weighting biases and deviations from normative rules. However, the hypothesis that one attribute is substituted for another in a judgment task – for example, representativeness for probability – can also be tested more directly. In the heuristic elicitation design, one group of respondents provides judgments of a target attribute for a set of objects and another group evaluates the hypothesized heuristic attribute for the same objects. The substitution hypothesis implies that the judgments of the two groups, when expressed in comparable units (e.g., percentiles), will be identical. Matching judgments support the substitution hypothesis, but other explanations for the match still need to be considered.
When to use it
Researchers can use heuristic elicitation to empirically test whether a particular heuristic (e.g., representativeness, availability) is driving judgments in a specific context.
Educators can demonstrate substitution bias by having students compare their own intuitive judgments with those of classmates judging a hypothesized heuristic attribute.
Policy makers can evaluate whether public risk perceptions are driven by availability or other heuristics by designing parallel surveys for target and heuristic attributes.
FROM THEORY TO EVERYDAY LIFE
In practice
Representativeness vs. Probability
As an illustration of the design, one group is asked to judge the probability that a person described as 'shy and withdrawn' is a librarian. Another group judges how representative that description is of a librarian. The heuristic elicitation design compares these two sets of judgments to see if probability is being replaced by representativeness.
Availability and Frequency Estimation
To test whether ease of recall (availability) is substituted for frequency estimates, one group estimates the number of words starting with the letter 'r', while another rates how easily such words come to mind. If judgments align, it suggests availability is being used as a heuristic.
A MOMENT TO REFLECT
Put it to work.
Can you recall a personal decision where you might have substituted an easier attribute (like similarity or ease of recall) for a harder one (like probability or frequency)?
How could the heuristic elicitation design be applied to test for substitution in a domain outside of psychology, such as economics or law?
What are the potential pitfalls of assuming that matching judgments in comparable units proves substitution?
CONTINUE THE CONVERSATION
Source texts
- Heuristics and Biases The Psychology of Intuitive Judgment (Thomas Gilovich, Dale Griffin, Daniel Kahneman)