Strength of Evidence
Banasiewicz proposes ranking evidence by objectivity and relevance to a decision, then giving each successive rank twice the weight of the previous one. The weights are normalized to total about 100%; their usefulness depends on how the evidence is ranked.
THE IDEA
Strength of Evidence
Banasiewicz proposes a hierarchy for combining evidence that points toward different conclusions. Each type is weighted based on its objectivity and decision specificity—the more objective and on-point, the higher the weight. Weights can be assigned using the doubling rule: rank-order evidence types from least to most objective/specific, then assign weights such that each higher-ranked type's weight is double the lower-ranked type, and all weights sum to ~100%.
Banasiewicz connects the doubling rule with just noticeable difference, the minimum detectable change. This is his rationale for the convention; noticing a difference does not itself establish an evidence source’s proper weight.
Meta-categories of Evidence
- Experiential evidence (less objective/decision-specific): overall weight ~33%
- Empirical evidence (more objective/decision-specific): overall weight ~67%
Within empirical evidence, four narrower types exist:
- Operational data (typically most objective/specific)
- Expert judgment
- Norms & standards
- Theoretical research
Using the doubling rule, the weakest-to-strongest weight assignment is: 7% – 13% – 27% – 53% (for four evidence types).
These weights can be combined via weighted linear pooling (WLP), generalizing the process of pooling expert opinions.
Note: The hierarchy is context-dependent; rank-ordering should follow careful assessment of each evidence type's informational content and decision-guiding value.
When to use it
Resolving conflicting data in business strategy formulation (e.g., market entry, product launch)
Prioritizing evidence types in policy-making or regulatory decisions
Weighting diagnostic criteria in complex medical or engineering assessments
Structuring multi-source intelligence analyses for national security or risk management
FROM THEORY TO EVERYDAY LIFE
In practice
Company Acquisition Decision
An acquisition team ranks theoretical research, norms & standards, expert judgment, and operational data from weakest to strongest for its particular decision. Assigning weights of 7%, 13%, 27%, 53% in that order gives operational data the largest weight. The team still needs to justify that ranking and check the quality of each source.
Policy Impact Assessment
A government agency evaluates a new policy using: empirical data (surveys), expert testimony, legal precedents, and academic research. The doubling rule helps them prioritize the most objective and decision-specific source (empirical data) by giving it the largest weight in their final recommendation.
A MOMENT TO REFLECT
Put it to work.
Think of a recent decision where you had conflicting evidence. How could rank-ordering sources by objectivity and decision-specificity change which evidence you prioritized?
Identify a situation where you might assign different evidence types to the same rank order because of context-specific factors. What makes that situation different?
What reasons support the weights you give different evidence sources? Would a different reasonable set of weights change your decision?
CONTINUE THE CONVERSATION
Source texts
- Evidence-Based Decision-Making How to Leverage Available Data and Avoid Cognitive Biases (Andrew D. Banasiewicz)