Systematic Review

Systematic Review

A systematic review asks a defined research question and uses an explicit plan to find, assess and synthesize relevant studies. A narrative review may draw on studies selected through an author’s knowledge and interests. A systematic review makes the selection process more transparent through search methods and criteria that other researchers can examine.

The Cochrane Collaboration, established in 1993, is a prominent example in health care. Its reviews bring together research on particular questions so practitioners can assess a body of evidence rather than rely on an isolated finding. Systematic methods aim to reduce bias; they do not remove the need to judge the quality and relevance of the underlying studies.

The process can be described in four steps:

• Ask an answerable question.
• Acquire the relevant research evidence.
• Appraise its quality.
• Aggregate and interpret the findings.

The search strategy, eligibility criteria and appraisal methods should make clear how the reviewers reached their conclusions. Reviewers also need to consider whether studies address sufficiently similar questions to be combined. Differences in definitions, settings or methods may explain why apparently related studies disagree.

Banasiewicz argues that this is a particular challenge in management research, where theoretical approaches and measures can vary widely. He also criticizes the gap between academic work and practitioners’ needs. Those concerns are reasons to examine applicability carefully, rather than dismiss an entire field of evidence.

The Profit Impact of Market Strategies database, or PIMS, illustrates a different kind of research resource. The source describes a long-term study of business-unit performance covering about 3,000 companies across industries. It began at General Electric in the mid-1960s, was briefly hosted by Harvard University in the 1970s, and later developed through the Strategic Planning Institute. A database can support many studies without being a systematic review: it supplies data rather than critically synthesizing all relevant studies on a defined question.

For a management decision, review findings can be considered alongside operational data and experience. Ask whether the research concerns a comparable organization, intervention and outcome. Academic findings may reveal something local experience misses, while local evidence may show why a result does not transfer easily.

Meta-analysis and practical importance

A systematic review may include a meta-analysis, which statistically combines results from multiple studies. Pooling is useful only when the studies and methods make the combined estimate meaningful. A review can still be systematic when quantitative pooling is inappropriate, and a meta-analysis is not automatically a systematic review.

An effect size describes the magnitude of a difference or association. This helps answer a practical question: is the estimated effect large enough to matter for the decision? Statistical significance alone does not answer that question or guarantee that a finding is free of bias or error. Consider the effect estimate, its uncertainty, the quality of the evidence and the consequences of applying it.