Take The Best Heuristic

Take the Best checks cues in order of their predictive validity and stops at the first cue that distinguishes the two objects being compared. The cue’s direction determines the prediction. If no cue distinguishes them, the heuristic guesses. The comparison below concerns the effort required to learn and apply that rule.

How Much Information Processing Is Performed?

In the studies summarized here, Take the Best and Minimalist stopped searching after looking up fewer than one-third of the available cues on average. That measures information use. To measure computational effort, we also need to count the operations used in training and prediction.

Let us begin by measuring the amount of learning required by the heuristics to build their models in order to perform their predictions later. We can use the suggestion of Newell and Simon (1972) and Payne, Bettman, and Johnson (1990) – count the number of elementary information processing (EIP) units necessary for the training phase. These EIPs include addition, subtraction, multiplication, division, comparison of two numbers, reading a number, writing a number, and so on. For each such operation, we count 1 unit. These EIP units are easy to count, and Payne, Bettman, and Johnson (1990) present experimental evidence that they are a reasonable estimate of the cognitive effort involved in executing a particular choice strategy in a specific task environment. For our tasks, the number of EIPs required depends on N, the number of objects, and M, the number of cues in the data set. Approximate EIP counts for any N and M, along with the specific values for predicting homelessness (N = 50, M = 6), show that fast and frugal heuristics require dramatically fewer calculations in the training phase than multiple regression. The training phase of Take The Best and Minimalist involves orders of magnitude fewer operations than the regression’s lower bound of roughly 20,000 EIPs (even without invertibility and computer overflow checks). In practical applications, fast and frugal heuristics might require as little as 1/100 of the operations. Note that we differ from earlier theorists such as Dawes (1979) in including learning the direction of cues as a real problem; other theorists have assumed this is known already, making fast and frugal heuristics even simpler.

Of course, learning a model of the data is only the first step. Implementing the heuristic has a cost, too. The number of EIPs in the test phase reveals that the heuristics in these comparisons use fewer operations than the more complex strategies, because they examine fewer cues and perform fewer calculations on them. Because fast and frugal heuristics generally do not use all of the available cues, we also need to consider the actual number of cues looked up. For example, Take the Best uses, on average, only 2.4 cues for predicting homelessness. The test-phase EIPs show that the cue-based predictions of Minimalist and Take the Best are about 5 times simpler than the simplest linear model, Dawes' Rule, and about 10 times simpler than Multiple Regression. They are faster even if some processing is done in parallel; under parallel cue validation, the total calculations would remain the same but would be completed more quickly, and they would still be more frugal.

Yet, how well do these computationally minimal heuristics actually predict? Performance on predicting homelessness, measured by the percentage of correct pairwise inferences, offers initial answers. When the test set coincides with the training set (fitting), Take the Best, which uses only 2.4 cues on average, scores higher than Dawes’ Rule, which uses all six cues, and is almost as accurate as linear regression, which performs complex calculations on all cues. In fact, across 20 different real-world environments, Take the Best competes favorably not only with Dawes’ Rule and multiple regression but also with much more sophisticated methods like Bayesian networks. For homelessness prediction, when the test set equals the training set, Take the Best is about 8 percentage points less accurate than a Bayesian network, but it is only 2 percentage points behind that network when the test set is different (cross-validation). In the fish fertility data, the large dataset produced stable results for all methods under cross-validation. For predicting professors’ salaries, Take the Best was 4 percentage points behind the Bayesian network in same-set tests but nearly matched it in cross-validation. Across these studies, a consistent pattern emerges: complex models like Bayesian networks and multiple regression achieve higher accuracy when fitting known data, but their advantage shrinks to as little as 1–2 percentage points when generalizing to new data. For the homelessness data, a Profile Memorization Method stores the cue profiles and outcomes but omits city names. It reaches only 82% correct, rather than 100%, because some cities with the same recorded cues have different outcomes. This limits what those cues can distinguish; it does not establish that homelessness is inherently unpredictable with other information.

Across the reported cross-validation comparisons, Take the Best was more accurate than regression on average, despite its lower computational cost.

One explanation is that Take the Best estimates relatively little: it used only 2.4 cues on average in the homelessness task. If the strongest cues and their ranking remain stable across samples, they can support useful predictions without estimating all cue weights and correlations. More elaborate models can overfit small training sets. This advantage depends on the data: a stopping rule can also discard useful evidence, so its accuracy should be assessed on held-out cases.