Hypothesis Testing Technique (Matrix Analysis)
A hypothesis matrix compares competing explanations with the available evidence. It highlights contradictions, identifies gaps, and helps investigators decide what information would distinguish the remaining possibilities.
THE IDEA
Introduction
The hypothesis-testing technique is a structured approach to evaluate competing explanations or hypotheses by systematically comparing each hypothesis against available evidence. It makes the comparison explicit and can expose confirmation bias, though its usefulness still depends on the hypotheses, evidence, and assumptions entered.
Steps
- Generate hypotheses: List all plausible explanations for the event or problem.
- Construct a matrix: Create a matrix with hypotheses as columns and significant evidence as rows.
- List significant evidence: Identify key facts that can help differentiate between hypotheses.
- Test consistency: For each hypothesis, determine whether the evidence is consistent (+), inconsistent (-), or neutral (0).
- Refine the matrix: Add or reword hypotheses, or delete evidence that is consistent with all hypotheses.
- Evaluate each hypothesis: Re-examine inconsistent evidence; consider rejecting hypotheses when reliable evidence contradicts them, after checking the assumptions behind the contradiction.
- Rank remaining hypotheses: Order by weakness of inconsistent evidence.
- Review the conclusion: Check assumptions, evidence reliability, and whether an important alternative has been omitted.
Example: Lost Data Exercise
Scenario: Sarah, an advertising agency president, learns that critical files for a client (Anderson) have been deleted from the computer and backup. Suspects are five employees: Gerald, Sam, Edward, Lisa, and Chuck. The deletion occurred between 4 PM and midnight, and no outside access was possible.
Evidence:
- The perpetrator was alone in the office at some point.
- The perpetrator is computer-savvy (can operate the computer to delete files).
- The perpetrator knew the password to the backup system.
Hypotheses: Each employee is a suspect.
Matrix Testing:
- Sam: Never alone, not computer-savvy, didn’t know password → eliminated.
- Edward: Never alone, didn’t know password → eliminated.
- Lisa: Never alone, didn’t know password → eliminated.
- Gerald: Was alone, computer-savvy, knew password → possible.
- Chuck: Was alone, computer-savvy, knew password → possible.
Conclusion: Gerald and Chuck are the remaining suspects; no further evidence distinguishes them in this exercise.
The matrix narrows the possibilities under the exercise’s assumptions; it does not establish which remaining person is responsible.
When to use it
Investigating a security breach or data loss in an organization.
Troubleshooting a technical failure with multiple potential causes.
Making a strategic business decision when several competing explanations exist.
Solving a mystery or conducting an intelligence analysis.
FROM THEORY TO EVERYDAY LIFE
In practice
Lost Data at the Advertising Agency
Sarah's agency lost critical client files. Using hypothesis testing, she listed five employee suspects, identified key evidence (alone, computer-savvy, password knowledge), and eliminated three suspects, leaving two plausible candidates.
Real-World Application: Medical Diagnosis
A doctor considers multiple possible diagnoses (hypotheses) for a patient's symptoms. She lists evidence (test results, history) and tests each diagnosis for consistency, ruling out those that contradict key findings, narrowing to the most likely condition.
A MOMENT TO REFLECT
Put it to work.
Think of a problem you are currently facing. List at least three possible explanations and evaluate them against available evidence using this technique.
What cognitive biases might cause you to prematurely reject a valid hypothesis? How does this technique help mitigate that?
How could this hypothesis-testing matrix be adapted for group decision-making to ensure all perspectives are considered?
CONTINUE THE CONVERSATION
Source texts
- The Thinkers Toolkit by Morgan D Jones