Scenario Planning

Scenario planning in practice

A few years ago, I consulted with a national nonprofit organization, After-School All-Stars (ASAS), to work with them on incorporating scenario planning into their budgeting.* At the time described, ASAS, founded in 1992 by Arnold Schwarzenegger, provided three hours of structured after-school programming for over 70,000 underserved youth in eighteen cities across the United States. They depend heavily on grants for funding and were struggling with budget planning, given the uncertainty in the grant award process. To help them with planning, I asked for a list of all their grant applications and how much each grant was worth. They provided me with a list of all their outstanding grant applications and the award amounts applied for. I told them that I didn’t see how much each grant was worth in the information they provided. They pointed to the column of the award amounts sought. At that point, I realized we were working from different ideas about how to determine worth. The misunderstanding came from the disconnect between the expected value of each grant and the amount they would be awarded if they got the grant.*

Coming up with the expected value of each grant involves a simple form of scenario planning: imagining the two possible futures that could result from the application (awarded or declined) and the likelihood of each future. For example, if they applied for a $100,000 grant that they would win 25% of the time, that grant would have an expected value of $25,000 ($100,000 × .25). If they expected to get the grant a quarter of the time, then it wasn’t worth $100,000; it was worth a quarter of $100,000. A $200,000 application with a 10% chance of success would have an expected value of $20,000. A $50,000 grant with a 70% chance of success would be worth $35,000. Comparing award amounts alone would put the $200,000 grant first, but the $50,000 grant has the highest expected receipts under these assumptions. Those receipts are an average across possible outcomes, not money guaranteed to arrive; application costs and restrictions also matter. ASAS recognized that uncertainty was causing them problems (to the point where they felt enslaved by it in their budgeting), but they hadn’t wrapped uncertainty into their planning or resource-allocation process. They were flying by the seat of their pants.

After I worked with the national office, ASAS made estimating the likelihood of getting each grant part of their planning. The benefits they got from scenario planning were immediate and substantial:

• They created a more efficient and productive work stack. Before doing this exercise, they naturally put a higher priority on applications seeking larger dollar amounts, executing those first, putting more senior staff on them, and being more likely to hire outside grant writers to get those applications completed. By shifting to thinking about the probability of getting each grant, they were now able to prioritize by how much the grant was actually worth to the organization in making these decisions. Thereafter, the higher value grants went to the top of the stack rather than just the grants with the higher potential awards.
• They could budget more realistically. They had greater confidence in advance estimates of the amount of money they could expect to receive.
• Because coming up with an expected value required estimating the likelihood of getting the grant, they increasingly focused on improving the accuracy of their estimates. This prompted them to close the loop by going back to the grantors. They had previously, after rejections, followed up with grantors. Because they were now focusing on checking and calibrating their probabilities, they expanded this to the grants they won. Overall, their post-outcome reviews focused on understanding what worked, what didn’t work, what was luck, and how to do better, improving both their probability estimates and the quality of their grant applications.
• They could think about ways to increase the probability of getting grants and commit to those actions.
• They were less likely to fall prey to hindsight bias because they had considered in advance the probability of getting or not getting the grant.
• They were less likely to fall prey to resulting because they had evaluated the decision process in advance of getting or not getting the grant.
• Finally, because of all the benefits ASAS received from incorporating scenario planning into budgeting and grant applications, they expanded the implementation of this type of scenario planning across departments, making it part of their decision-making culture.

Grant prospecting is similar to sales prospecting, and this process can be implemented for any sales team. Assign probabilities for closing or not closing sales, and the company can do better at establishing sales priorities, planning budgets and allocating resources, evaluating and fine-tuning the accuracy of its predictions, and protecting itself against resulting and hindsight bias.

A more complex version of scenario planning occurs when the number of possible futures multiplies and/or we go deeper into the tree, considering what we will do next in response to how things turn out and what the set of outcomes of that next decision is and so on.

Consider the scenario planning involved in Seahawks coach Pete Carroll’s much-criticized Super Bowl decision, trailing by four, twenty seconds remaining, one time-out, second-and-goal at the Patriots’ one-yard line. Carroll has two general choices, run or pass, and they in turn lead to multiple scenarios.

If Carroll calls for a run, these are the possible futures: (a) touchdown (immediate win); (b) turnover-fumble (immediate loss); (c) tackle short of the goal line; (d) offensive penalty; and (e) defensive penalty. Futures (c)–(e) branch into additional scenarios. The most likely failure scenario, by far, is that the runner is tackled before reaching the end zone. Seattle could stop the clock with its final time-out, but if they run the ball again and do not score, time will expire.

If Carroll calls for a pass, the possible futures are (a) touchdown (immediate success); (b) turnover-interception (immediate failure); (c) incomplete pass; (d) sack; (e) offensive penalty; and (f) defensive penalty. Again, the first two futures essentially end the game and the others branch off into additional play calling and additional outcomes.

The main difference between passing and running is that calling a pass likely gives Seattle a total of three plays to score, instead of two if Carroll calls a running play. An unsuccessful run would require that Seattle use their final time-out to stop the clock so they could run a second play. An incomplete pass would stop the clock and leave Seattle with a time-out and the chance to call those same two running plays. An interception, which negates the possibility of a second or third offensive play, is only a 2%–3% probability, a small price to pay for three chances to score rather than two. (A turnover caused by a fumble on a running play is 1%–2%.)*

Notice that the option of the extra play doesn’t reveal itself without this kind of scouting of the future. Even after the fact, with lots of time to analyze the decision, very few commentators saw this advantage.

The important thing is that we do better when we scout all these futures and make decisions based on the probabilities and desirability of the different futures. ASAS couldn’t guarantee it would get awarded every grant it applied for, but it could, through good process, make better decisions about which grants to prioritize and how much revenue to expect from the basket of grant proposals they submitted. Despite the outcry, I’d like to think that Pete Carroll didn’t lose much sleep over his decision to call for Wilson to pass.

Mapping possible futures gives us a record of what we considered before the result was known. We can use it both to plan a response and to review the reasoning afterward, rather than judging the decision solely by its outcome.