Belief-System Defenses

Belief-System Defenses: Discussion Following Table 42.2

Each defense invokes a potentially defensible reason for why relatively inaccurate experts should not abandon prior probability assessments that were conditional on their original understanding of underlying forces. Why change one’s mind about the validity of one’s pre-forecast conceptual framework when the necessary antecedent conditions for applying that framework were never fulfilled, exogenous shocks vitiated the ceteris paribus requirement for all fair tests of hypotheses, the predicted event almost occurred despite all these obstacles and still might, and prediction exercises are so riddled with indeterminacy as to be meaningless? This analysis suggests that the focus on whether experts got their Bayesian arithmetic right is misplaced. The more fundamental question concerns the defensibility of each belief-system defense. Let’s consider each in turn.

1) Nonfulfillment of antecedent and ceteris-paribus clauses. There are sound philosophy-of-science grounds for refusing to abandon a theory until one is convinced that the antecedent conditions for activating the relevant covering laws have been fulfilled and the ceteris paribus clause has been satisfied (Hempel, 1965). Experts often had reason for suspecting that the “fundamental forces” on which forecasts were initially predicated were not the same forces shaping events at the end of the forecasting period. Inherently unpredictable exogenous shocks had transformed the original meanings of key analytical concepts – rationality, economic determinism, the perceived interests of former apparatchiks or the white business elite – underlying forecasts. And background conditions – too numerous to specify in advance – occasionally took on extreme values that routed events down “unforeseeable” paths.

2) Close-call counterfactuals. Political outcomes often appear under the control of complex conjunctions of stochastic processes that could easily have yielded different effects. One’s reaction to this defense hinges on one’s metaphysical outlook. Laplacian determinists reject it out of hand, but if one accepts Steven Jay Gould’s (1995) provocative thought experiment in which, holding initial conditions constant, repeatedly re-running the tape of evolutionary history yields thousands of distinctive outcomes (with intelligent life one of the least likely), it is reasonable to talk about alternative histories “almost happening” and even to distinguish alternative histories on the basis of how “close” they came to happening. It is certainly not far-fetched to claim that the Parti Québecois almost won the second secessionist referendum. It is arguably not far-fetched to claim that hardline communists nearly reclaimed power in the late Gorbachev period (and might have had they been smarter and more cohesive) or that plans for European Monetary Union nearly collapsed in the wake of the 1992 currency crisis (and might have but for the obstinacy of key politicians such as Mitterrand and Kohl) or that the Gulf War of 1991 was almost averted (and would have been but for Saddam Hussein’s irrational unwillingness to undertake a partial preemptive withdrawal from Kuwait in the fall of 1990). Indeed, if we interpret the subjective-probability estimates attached to forecasts literally as “frequentist” claims about the distribution of possible worlds in repeated simulations of history, experts who assigned 80% confidence to “incorrect” forecasts may quite literally have been correct. Strange though it sounds, in the ontology of Lewis’ (1973) modal logic, it makes sense to say that in 80% of the Soviet Unions that were possible as of 1988, the predicted outcome occurred. We just happened to wind up in that unlikely subset of possible worlds in which communist control collapsed. Of course, this defense quickly wears thin: People will tire of hearing that the world they happen to inhabit is vanishingly improbable.

3) Off-on-timing. This defense is reminiscent of the probably apocryphal account of a Trotskyite rally at which the speaker declares Leon Trotsky to be a remarkably far-sighted man and offers as proof that “of all the predictions Trotsky has made not one has yet come true.” That said, this defense still merits a hearing. The domains in which the defense was most frequently invoked – Russia, the EMU, and Canada – were those in which experts were especially unsure, ex ante, of the temporal boundaries to place on their forecasts. At the time of this discussion, disagreement remained over whether the Russian communist party or another authoritarian force would triumph, and whether Canada and the EMU would dissolve. Posterity will ridicule us, not the experts, if we classify forecasts as categorically wrong that turn out to have been just a bit tardy.

4) Dismissing the Conceptual Significance of Prediction. This defense takes the indeterminacy argument to its ultimate extension. Rather than invoking indeterminacy in piecemeal fashion (a close-call counterfactual here or there), experts proclaim politics to be hopelessly “cloudlike,” disparage those who look for lawful regularities as subscribing to a naive “clocklike” perspective, and dismiss forecasting exercises as trivial at best or deeply misleading at worst (the winners deserve no more credit for good judgment than does the lucky roulette player who repeatedly bets on his favorite nephew’s birthdate). These experts are embracing a philosophical position with radical implications for the notion of good judgment. There should be as little consistency in who wins political forecasting tournaments as there is in forecasting tournaments for financial markets where random-walk models are widely held to hold (Malkiel, 1990).