Good Decisions and Bad Outcomes

Good Decisions Can Have Bad Outcomes

The maxim here is “Good decisions sometimes have bad outcomes.” It applies even when many or all of the available choices are likely to turn out badly. The chances that you will be pleased with the outcome of the decision are small, but analytic decision making can still be useful in guiding you to the best possible decision (or least bad one) given the choices you face.[54]

Some good decisions, indeed, are likely to lead to a bad outcome. Suppose you invest $1,000 in a talented close friend’s startup company at the same price he is selling to his immediate family. You recognize it has a good chance to go broke, say 90 percent, but if it succeeds your investment will be worth $50,000. You invest because the lottery is very favorable, and you can afford the loss of $1,000. The likelihood of a bad outcome from this good decision is 90 percent.

Alice Heath, a student of Richard’s at the Harvard Kennedy School and one of his teaching assistants at the time of this account, experienced this maxim very clearly when she started working with state child welfare agencies, whose mission is to prevent child abuse and neglect. The children and families they work with face very tough circumstances. Unfortunately, there is often no policy choice that a child welfare agency’s leadership can make that is likely to completely prevent abuse or neglect. “Completely preventing abuse or neglect would likely require draconian measures that would not be good for anyone. The best an agency can do is make the choice that has a higher probability of a better outcome relative to the other choices. Even with the best decisions there will still, sadly, be a high chance that some children suffer abuse and neglect. I have seen state legislators and commentators fail to understand this idea over and over, reading every tragic incident as a decision-making failure rather than the result of a set of choices where the best option is not a good option. As a result, state child welfare directors too often have very short terms and agencies lack stable leadership, which only makes things worse for the children and families who need help.”

Another illustration of this maxim comes from Al Carnesale. Al is an expert on national security policy, a decision tree maven, and one of Richard’s former colleagues. He had stints as chancellor at UCLA, provost of Harvard University, and dean of the Harvard Kennedy School. Throughout the Cold War (and into the period described here) the United States’ basic strategy to avoid nuclear war has been “nuclear deterrence.” In essence, the United States maintains the capability to retaliate against a nuclear attack with a counterattack so devastating that no nation will choose to attack it. The Soviet Union, and then Russia, also chose the nuclear deterrence strategy.

Al observes: “In short, both the U.S and the Soviet/Russian governments decided that the best way to avoid nuclear war was to maintain thousands of nuclear weapons aimed at the other side, with many on hair-trigger alert. The outcome of this decision thus far has been good: there has been no nuclear war between the two sides. Does this imply that this decision has been a good one? Not necessarily. It might well have been a bad decision and we’ve been just plain lucky.” Al notes that there may have been alternative strategies to avoid nuclear war that had a lower probability of a poor outcome, and that these alternatives might still exist now. He cautions against relying on good luck.

For anyone who knows Richard, one of the best-known illustrations of this maxim is an extremely challenging health decision he and his wife Sally had to make in 1995. That year, Sally had received a diagnosis of stage 3 breast cancer – a cancer that had spread to the lymph nodes. No option was attractive; each involved both arduous treatment and a significant chance of death.

Sally had been told that she had eight positive nodes. One doctor told her that if she had at least ten positive nodes, she would be eligible for a randomized controlled trial (RCT) where one arm was high-dose chemotherapy paired with a bone marrow transplant (BMT). Richard, just to expand options, asked the doctor to get a recount of the nodes. There turned out to be eleven, as some were clustered together and were hard to count. This raised count was bad news, but it made Sally eligible for the trial.

Her doctors observed: “We do not like to do bone marrow transplants, since they have a 4 percent treatment mortality.” Richard and Sally asked: “What gain does a BMT offer in survival probability, assuming one survives the treatment?” The doctors responded: “We do not know. That is why we are conducting this trial.” The results thus far were confidential. After pushing Sally’s doctors further, and having Richard read widely in the medical literature, including the application for the trial, and having him consult with oncologists at multiple institutions, they concluded that the expected long-term survival gain was 10 percent or more.

They learned three more things: An expert from Columbia University had written an article based on limited data that presented a graph suggesting that BMTs would provide better long-term survival, far past the five-year mortality threshold used in most studies. Second, Sally’s hospital had had 0 treatment fatalities to that point, not the overall 4 percent stated. Third, the sister of the statistician on the trial had recently received a BMT for breast cancer. Given all this information, Richard and Sally decided that she would apply to the trial. She did and was randomized to the BMT treatment.

The treatment was awful and put Sally within a whisker of death. Twenty years later, her oncologist stated that in fact the benefits were seen long after the treatment. At the time of this account, twenty-five years later, Sally was healthy. Both she and Richard recognize that they were lucky. They can’t be confident that they made the right decision. What they do know is that they made the right decision on the basis of the scraps of information they had at the time.[55]

Part of what makes Richard extraordinary is that even in the face of such a terrible choice in his personal life, when many of us familiar with this maxim would retreat to our base instincts, he is able to bring his analytical skills to make better decisions. As he thought at the time: “I have been training for this decision my entire life.”