Skip to main content
Mental Models

How to 'Prove' Things

Taleb criticizes models whose elegant proofs depend on assumptions that fit the world poorly. His skeptical-empiricist alternative emphasizes testing premises, learning from practice, and seeking decisions that hold up across a broad range of outcomes.

THE IDEA

How to “Prove” Things

Merton the younger is a representative of the school of neoclassical economics, which, as we have seen with LTCM, represents most powerfully the dangers of Platonified knowledge.* Looking at his methodology, I see the following pattern. He starts with rigidly Platonic assumptions, completely unrealistic—such as the Gaussian probabilities, along with many more equally disturbing ones. Then he generates “theorems” and “proofs” from these. The math is tight and elegant. The theorems are compatible with other theorems from Modern Portfolio Theory, themselves compatible with still other theorems, building a grand theory of how people consume, save, face uncertainty, spend, and project the future. He assumes that we know the likelihood of events. The beastly word equilibrium is always present. But the whole edifice is like a game that is entirely closed, like Monopoly with all of its rules.

A scholar who applies such methodology resembles Locke’s definition of a madman: someone “reasoning correctly from erroneous premises.”

Now, elegant mathematics has this property: it is perfectly right, not 99 percent so. This property appeals to mechanistic minds who do not want to deal with ambiguities. Unfortunately you have to cheat somewhere to make the world fit perfect mathematics; and you have to fudge your assumptions somewhere. We have seen with the Hardy quote that professional “pure” mathematicians, however, are as honest as they come.

So where matters get confusing is when someone like Merton tries to be mathematical and airtight rather than focus on fitness to reality.

This is where you learn from the minds of military people and those who have responsibilities in security. They do not care about “perfect” ludic reasoning; they want realistic ecological assumptions. In the end, they care about lives.

I mentioned in Chapter 11 how those who started the game of “formal thinking,” by manufacturing phony premises in order to generate “rigorous” theories, were Paul Samuelson, Merton’s tutor, and, in the United Kingdom, John Hicks. These two wrecked the ideas of John Maynard Keynes, which they tried to formalize (Keynes was interested in uncertainty, and complained about the mind-closing certainties induced by models). Other participants in the formal thinking venture were Kenneth Arrow and Gerard Debreu. All four were Nobeled. All four were in a delusional state under the effect of mathematics—what Dieudonné called “the music of reason,” and what I call Locke’s madness. All of them can be safely accused of having invented an imaginary world, one that lent itself to their mathematics. The insightful scholar Martin Shubik, who held that the degree of excessive abstraction of these models, a few steps beyond necessity, makes them totally unusable, found himself ostracized, a common fate for dissenters.*

If you question what they do, as I did with Merton Jr., they will ask for “tight proof.” So they set the rules of the game, and you need to play by them. Coming from a practitioner background in which the principal asset is being able to work with messy, but empirically acceptable, mathematics, I cannot accept a pretense of science. I much prefer a sophisticated craft, focused on tricks, to a failed science looking for certainties. Or could these neoclassical model builders be doing something worse? Could it be that they are involved in what Bishop Huet calls the manufacturing of certainties?

TABLE 4: TWO WAYS TO APPROACH RANDOMNESS
Skeptical Empiricism and the a-Platonic School The Platonic Approach
Interested in what lies outside the Platonic fold Focuses on the inside of the Platonic fold
Respect for those who have the guts to say “I don’t know” “You keep criticizing these models. These models are all we have.”
Fat Tony Dr. John
Thinks of Black Swans as a dominant source of randomness Thinks of ordinary fluctuations as a dominant source of randomness, with jumps as an afterthought
Bottom-up Top-down
Would ordinarily not wear suits (except to funerals) Wears dark suits, white shirts; speaks in a boring tone
Prefers to be broadly right Precisely wrong
Minimal theory, considers theorizing as a disease to resist Everything needs to fit some grand, general socioeconomic model and “the rigor of economic theory;” frowns on the “descriptive”
Does not believe that we can easily compute probabilities Built their entire apparatus on the assumptions that we can compute probabilities
Model: Sextus Empiricus and the school of evidence-based, minimum-theory empirical medicine Model: Laplacian mechanics, the world and the economy like a clock
Develops intuitions from practice, goes from observations to books Relies on scientific papers, goes from books to practice
Not inspired by any science, uses messy mathematics and computational methods Inspired by physics, relies on abstract mathematics
Ideas based on skepticism, on the unread books in the library Ideas based on beliefs, on what they think they know
Assumes Extremistan as a starting point Assumes Mediocristan as a starting point
Sophisticated craft Poor science
Seeks to be approximately right across a broad set of eventualities Seeks to be perfectly right in a narrow model, under precise assumptions

Let us see.

Skeptical empiricism advocates the opposite method. I care about the premises more than the theories, and I want to minimize reliance on theories, stay light on my feet, and reduce my surprises. I want to be broadly right rather than precisely wrong. Elegance in the theories is often indicative of Platonicity and weakness—it invites you to seek elegance for elegance’s sake. A theory is like medicine (or government): often useless, sometimes necessary, always self-serving, and on occasion lethal. So it needs to be used with care, moderation, and close adult supervision.

The distinction in the above table between my model modern, skeptical empiricist and what Samuelson’s puppies represent can be generalized across disciplines.

* This is a simple illustration of the general point of this book in finance and economics. If you do not believe in applying the bell curve to social variables, and if, like many professionals, you are already convinced that “modern” financial theory is dangerous junk science, you can safely skip this chapter.

* Granted, the Gaussian has been tinkered with, using such methods as complementary “jumps,” stress testing, regime switching, or the elaborate methods known as GARCH, but while these methods represent a good effort, they fail to address the bell curve’s fundamental flaws. Such methods are not scale-invariant. This, in my opinion, can explain the failures of sophisticated methods in real life as shown by the Makridakis competition.

* More technically, remember my career as an option professional. Not only does an option on a very long shot benefit from Black Swans, but it benefits disproportionately from them—something Scholes and Merton’s “formula” misses. The option payoff is so powerful that you do not have to be right on the odds: you can be wrong on the probability, but get a monstrously large payoff. I’ve called this the “double bubble”: the mispricing of the probability and that of the payoff.

* I am selecting Merton because I found him very illustrative of academically stamped obscurantism. I discovered Merton’s shortcomings from an angry and threatening seven-page letter he sent me that gave me the impression that he was not too familiar with how we trade options, his very subject matter. He seemed to be under the impression that traders rely on “rigorous” economic theory—as if birds had to study (bad) engineering in order to fly.

* Medieval medicine was also based on equilibrium ideas when it was top-down and similar to theology. Luckily its practitioners went out of business, as they could not compete with the bottom-up surgeons, ecologically driven former barbers who gained clinical experience, and after whom a-Platonic clinical science was born. If I am alive, today, it is because scholastic top-down medicine went out of business a few centuries ago.

When to use it

  • Evaluating financial models: A trader should be skeptical of models that assume Gaussian distributions for rare events.

  • Policy-making: A government should avoid basing economic policy on equilibrium models that ignore Black Swans.

  • Scientific research: Checking whether a model’s assumptions and predictions fit observations when applying it to a complex system.

  • Personal decision-making: When facing uncertainty, it's better to have a robust strategy that works across multiple scenarios than a precise strategy that fails in extreme events.

FROM THEORY TO EVERYDAY LIFE

In practice

01

Merton's Neoclassical Economics

Taleb uses Merton’s neoclassical models to argue that mathematically consistent results can still be unreliable guides when the assumptions do not fit the situation. His criticism concerns the connection between the model and the world it is used to describe.

02

The Skeptical Empiricist Alternative

A practitioner focuses on messy empirical mathematics, avoids over-reliance on theories, and aims to be broadly right rather than precisely wrong, as exemplified by the author's own career in options trading.

✳

A MOMENT TO REFLECT

Put it to work.

  1. How might the Platonic approach lead to false confidence in real-world decisions?

  2. In what areas of your own work do you rely on elegant theories rather than messy empirical evidence?

  3. Can you think of a time when being precisely wrong was worse than being broadly right?

Connected ideas

Another lens can bring the same problem into focus.

Often confused with

CONTINUE THE CONVERSATION

Source texts

  • The Black Swan The Impact of the Highly Improbable With a new section On Robustness and Fragility (Nassim Nicholas Taleb)
← More mental models