There are many types of reviews of the medical and public health evidence, each with its own benefits and challenges. In this blog post, we detail five key differences between a systematic review and other types of reviews, including narrative and comprehensive reviews.
First, we must define some terms. “Literature review” is a general term that describes a summary of the evidence on a certain topic. Literature reviews can be very simple or highly complex, and they can use a variety of methods for finding, assessing, and presenting evidence. A “systematic review” is a specific type of review that uses rigorous and transparent methods in an effort to summarize all of the available evidence with little to no bias. A good systematic review adheres to the international standards set forth in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 27-item checklist.1 Reviews that are less rigorous are often called “narrative,” “comprehensive,” or simply “literature reviews.”
So, what are the 5 key differences between a systematic review and other types of review?
1. The goal of the review
The goal of a literature review can be broad and descriptive (example: “Describe the available treatments for sleep apnea”) or it can be to answer a specific question (example: “What is the efficacy of CPAP for people with sleep apnea?”). The goal of a systematic review is to answer a specific and focused question (example: “Which treatment for sleep apnea reduces the apnea-hypopnea index more: CPAP or mandibular advancement device?”). People seeking to make evidence-based decisions look to systematic reviews due to their completeness and reduced risk of bias.
2. Searching for evidence
Where and how one searches for evidence is an important difference. While literature reviews require only one database or source, systematic reviews require more comprehensive efforts to locate evidence. Multiple databases are searched, each with a specifically tailored search strategy (usually designed and implemented by a specialist librarian). In addition, systematic reviews often include attempts to find data beyond typical databases. Systematic reviewers might search conference abstracts or the web sites of professional associations or pharmaceutical companies, and they may contact study authors to obtain additional or unpublished data. All of these extra steps reflect an attempt to minimize bias in the summary of the evidence.
3. Assessing search results
In a systematic review, the parameters for inclusion are established at the start of the project and applied consistently to search results. Usually, such parameters take the form of PICOs (population, intervention, comparison, outcomes). Reviewers hold search results against strict criteria based on the PICOs to determine appropriateness for inclusion. Another key component of a systematic review is dual independent review of search results; each search result is reviewed by at least two people independently. In many other literature reviews, there is only a single reviewer. This can result in bias (even if it is unintentional) and missed studies.
4. Summary of findings
In a systematic review, an effort is usually made to assess the quality of the evidence, often using risk of bias assessment, at the study level and often across studies. Other literature reviews rarely assess and report any formal quality assessment by individual study. Risk of bias assessment is important to a thorough summary of the evidence, since conclusions based on biased results can be incorrect (and dangerous, at worst). Results from a systematic review can sometimes be pooled quantitatively (e.g., in a meta-analysis) to provide numeric estimates of treatment effects, for example.
5. Utility of results
Due to the rigor and transparency applied to a systematic review, it is not surprising that the results are usually of higher quality and at lower risk of bias than results from other types of literature review. Literature reviews can be useful to inform background sections of papers and reports and to give the reader an overview of a topic. Systematic reviews are used by professional associations and government agencies to issue guidelines and recommendations; such important activities are rarely based on a non-systematic review. Clinicians may also rely on high quality systematic reviews to make evidence-based decisions about patient care.
Each type of review has a place in the scientific literature. For narrow, specific research questions, a systematic review can provide a thorough summary and assessment of all of the available evidence. For broader research questions, other types of literature review can summarize the best available evidence using targeted search strategies. Ultimately, the choice of methodology depends on the research question and the goal of the review.
[1] Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097.