Delphi Approximations

Delphi Approximations

Delphi[4] method is a systematic, interactive forecasting technique which relies on a panel of independent experts, originally developed at the beginning of the cold war to forecast the impact of technology on warfare. The method was conceived as a solution to the problem of developing credible estimates without the benefit of empirical evidence, in response to which Douglas Aircraft Company started Project RAND (R esearch AN d D evelopment, now an independent nonprofit think tank) to study ‘the broad subject of inter-continental warfare other than surface’. Finalized as a methodology by the late 1950s, Delphi method is based on the principle that forecasts from a structured group of experts are more accurate than those from unstructured groups or individuals; since its inception, the method has been adapted to a variety of business applications, including new product design and marketing.

It works as follows: The carefully selected experts answer questionnaires in two or more rounds. After each round, a facilitator provides an anonymous summary of the experts’ forecasts from the previous round as well as the reasons they provided for their judgments, and participants are encouraged to revise their earlier answers in light of the replies of other members of the group. During that process, which is iterative, the range of answers tends to decrease, and the group begins to converge toward a singular answer. Finally, the process is stopped after a predefined stop criterion, such as the number of rounds or the achievement of consensus or the stability of results, and the mean or median scores of the final round determine the outcomes.

The method uses successive approximations: participants revise estimates as they consider additional information and competing reasons. Within the 3E framework, the resulting estimates are called Delphi approximations, as summarized in [Figure 9.2]. This process can expose weaknesses in an estimate, but agreement among experts does not independently validate it.

[FIGURE 9.2 Delphi Approximations.]

The author also assesses similarity among experts’ initial estimates, calling it inter-expert reliability (IER). One proposed approach correlates experts’ responses across items. Similar initial responses indicate agreement, though correlation alone does not show that the estimates are accurate or equal in level. The referenced Appendix B provides the author’s additional criteria.