Skip to main content
Cognitive Biases

Noticing Bias

Motivated reasoning biases our judgment without our awareness, akin to a magician's 'forcing' trick. We apply different standards to identical situations based on our affiliation or desired outcome, yet feel entirely objective because we never consider the counterfactual.

THE IDEA

Noticing Bias

To grasp how insidious motivated reasoning is, it helps to know a little magic.

One of the essential tools in a magician’s tool kit is a form of manipulation called forcing. In its simplest form, forcing works like this: The magician places two cards facedown in front of you. In order to make his trick succeed, he needs you to end up with the card on the left. He says: “Now we’re going to remove one of these cards—please pick one.”

If you point to the card on the left, he says, “Okay, that one’s yours.” If you point to the card on the right, he says, “Okay, we’ll remove that one.” Either way, you end up holding the card on the left, feeling like you chose it of your own free will. If you could see both of those possible scenarios at once, the trick would be obvious. But because you end up in only one of those worlds, you never realize.

Forcing is what your brain is doing to get away with motivated reasoning while still making you feel like you’re being objective. Suppose a Democratic politician gets caught cheating on his wife, but a Democratic voter doesn’t consider that a reason not to vote for him: “What he does in his private life is his own business,” she reasons. However, if the adulterous politician had been a Republican, she would instead have thought, “Adultery is a sign of poor character—that shows he’s not fit to govern.”

If the Democratic voter could see the way she would have reacted in that counterfactual world and compare it to her reaction in the actual world, the influence of her motivations would be obvious to her. But because she only ever sees one of those worlds, she never realizes that she’s being anything less than dispassionate.

It’s easiest for your brain to pull the “forcing” trick on topics you’ve never considered before, because you have no preexisting principles to get in the way of choosing whichever answer is convenient for you in the case at hand. You may have already formed opinions about how harshly to judge adultery, so how about this example instead: If you get sued and you win the case, should the person who sued you pay your legal costs? If you’re like most people (85 percent, in one study), your answer is yes. After all, if you’re falsely accused of something, why should you be out thousands of dollars in lawyers’ fees? That wouldn’t be fair.

However, when the question in that study was slightly reworded—“If you sue someone and you lose the case, should you pay his costs?”—only 44 percent of people said yes. Imagining yourself in the role of the person who sued and lost brings to mind alternate arguments. For example, you might have lost simply because the other side is wealthy and can afford better lawyers. It’s not fair to discourage victims from suing just because they can’t afford to lose, right?

Both the arguments for and against the “loser pays” policy have at least some merit. But which one comes to mind will depend on whether you’re the plaintiff or the defendant—and it will likely never occur to you that you could have thought of an opposing argument if you had been on the other side of the case.

When to use it

  • Negotiations: Realizing that both sides might genuinely believe their offer is fair, because they are only considering the factors that benefit their own position.

  • Product Design: A product manager dismisses user complaints because they are 'power users', but would take the same complaints very seriously if they came from their target demographic.

  • Performance Reviews: A manager overlooks a favorite employee's mistakes while heavily scrutinizing a less favored employee's identical performance.

  • Public Policy: Citizens judging tax policies based on how they affect their own income bracket vs. a broader economic principle of fairness.

FROM THEORY TO EVERYDAY LIFE

In practice

01

Political Hypocrisy on Adultery

A Democratic voter might forgive a Democratic politician for adultery, saying 'his private life is his own business', but would condemn a Republican politician for the same act, citing 'poor character'. The voter feels objective in both cases because they never see the counterfactual scenario where their standards flip.

02

Loser Pays Legal Fees

In a study, 85% of people agreed that if they were sued and won, the loser should pay their costs. However, only 44% agreed that if they sued someone and lost, they should pay the winner's costs. People rationalized the same rule differently depending on their imagined role in the scenario.

✳

A MOMENT TO REFLECT

Put it to work.

  1. Can you recall a time you were absolutely certain of a position, but realized you might have argued the opposite if your personal stakes were different?

  2. How can you actively imagine the counterfactual scenarios to identify where your own 'forcing' mechanisms are at play in a current debate or decision?

  3. The text describes a legal cost study. Can you identify a political or social issue where the side you're on completely changes your assessment of fairness or a general principle?

Connected ideas

Another lens can bring the same problem into focus.

CONTINUE THE CONVERSATION

Source texts

  • The Scout Mindset (Julia Galef)
← More cognitive biases