Systematic Reviews and Scoping Reviews are types of literature reviews that use systematic methods to collect studies and each have their own complex steps. ‘New’ researchers will experience a steep learning curve and should be prepared to be trained in the art of searching before the review is conducted.
To help you understand the difference between a systematic or scoping review as well as provide you with a high-level overview of the work involved, please review the side-by-side comparison chart below.
| Systematic Review | Scoping Review | |
|---|---|---|
| Definition |
A systematic review is a type of literature review that uses systematic methods to collect secondary data, critically appraise research studies, and synthesize findings qualitatively or quantitatively to answer a specific research question. |
A scoping review differs from a systematic review in its purpose and aim. The purpose of a scoping review is to provide an overview of the available research evidence without producing a summary answer to a discrete research question. |
| Average Time to Complete Review | 6 months to 1.5 years | 9 months to 1 year |
| Minimum Number of People Require to Complete Review | 3 | 2 |
| Steps for Writing Your Review |
|
|
If performing a systematic review, we highly recommend that you complete all of the Cochrane Interactive Learning Modules before you begin.
Should you require support anytime throughout your research process and would like to book a research consult, please contact a member of the Western Libraries Systematic and Scoping Review Team.
If you determine a systematic or scoping review is NOT currently the right path for you and you need research support, please fill out this research consultation form.
Timeline of your project is key. One common mistake of researchers undergoing a Systematic or Scoping Review is an unrealistic project timeline. Please see Cochrane timeline below
Box 2.3.b: Timeline for a Cochrane review
|
Month Activity 1 – 2 Preparation of protocol. 3 – 8 Searches for published and unpublished studies. 2 – 3 Pilot test of eligibility criteria. 3 – 8 Inclusion assessments. 3 Pilot test of ‘Risk of bias’ assessment. 3 – 10 Validity assessments. 3 Pilot test of data collection. 3 – 10 Data collection. 3 – 10 Data entry. 5 – 11 Follow up of missing information. 8 – 10 Analysis. 1 – 11 Preparation of review report. 12 – Keeping the review up-to-date. |
| Month* | Task | Description | Stage |
|---|---|---|---|
| 1 - 2 | Formulate Research Question | Decide on the research question for the review | Preparation |
| 1 - 2 | Investigate For Previous SRs | Search published and unpublished studies (SRs) that answer the same question | Preparation |
| 1 - 2 | Devise Search Strategy | Decide on databases, keywords, and subject headings to find all relevant trials | Preparation |
| 2 - 3 | Write/Register Protocol | Provide an objective, reproducible, sound methodology for peer review | Preparation |
| 3 - 5 | Primary Search | Aim to find all relevant citations even if many irrelevant ones are included | Retrieval |
| 3 - 5 | Grey Literature Search | If applicable - Review of not formally published articles (conference proceedings, government publications, etc.) | Retrieval |
| 4 - 5 | De-Duplicate | Remove identical citations | Retrieval |
| 5 - 8 | Title & Abstract Screening | Screen titles and abstracts, remove irrelevant articles based on inclusion/exclusion criteria | Appraisal |
| 5 - 8 | Obtain Full Text | Download, request copies from authors, inter-library loans, etc. | Retrieval |
| 5 - 8 | Full Text Screening | Exclude irrelevant articles based on inclusion/exclusion criteria | Appraisal |
| 9 | Supplemental Search | Follow citations from included articles to potentially locate additional articles. New articles must go through screening process | Retrieval |
| 9 - 11 | Data Extraction | Obtain the necessary information about study characteristics and findings from the included studies. | Synthesis |
| 9 - 11 | Data Synthesis | Convert extracted data to common representations (usually average and SD) | Synthesis |
| 9 - 11 | Data Analysis | Using qualitative or quantitative data, combine results from all included trials | Synthesis |
| 11 | Re-Run Primary Search | Repeat the search to find new literature published since initial search. New articles must go through screening process | Retrieval |
| 1 - 12* | Write Up Review | Produce and publish final report. Some elements should be written throughout different stages of the review | Write-up |
| * This timeline is only a guide. Systematic Reviews can take up to 2 years depending on the complexity of your topic and the time and resources available to your team | |||
The information below has been adapted from: Cochrane’s Handbook for Systematic Reviews of Interventions & McGill Library’s Systematic Reviews, Scoping Reviews, and other Knowledge Syntheses Guide & UofT Libraries’ Knowledge syntheses: Systematic & Scoping Reviews, and other review types Guide
Formulating a well-constructed research question is essential for a successful review. You should have a draft research question before you choose the type of review that you will conduct to ensure a systematic or scoping review is the appropriate choice.
A clearly defined question will allow you to focus your search so that it is more efficient and effective, make it easier to find and combine appropriate terms, and help identify relevant results.
To define and focus your question you can utilize a concept mapping method. Please see our How to Develop a Research Question video as an introductory resource on developing research questions.
Once you have a clearly defined research question, it is important to make sure your question has not already been recently and successfully undertaken. This means it is important to find out if there are other reviews that have been published or that are in the process of being published on your topic.
Even if you do find another review or synthesis on your topic, it may be sufficiently out of date or you may find other defendable reasons to perform it again. In addition, looking at other reviews published around your topic may also help you refocus your question or redirect your research toward other gaps in the literature.
To find published reviews or protocols on your topic, you can check resources such as:
PROSPERO - to find protocols
Cochrane Library – to find systematic reviews
Other databases or journals that index systematic/scoping reviews or protocols
When devising a search strategy, remember it is supposed to be thorough, objective and reproducible. It involves searching more than one database and using a combination of keywords as well as subject headings.
It is recommended to consult a librarian at this stage of your review.
For more information on devising a comprehensive search strategy visit our Searching Techniques Page which can be found within our Systematic and Scoping Review Guide.
What is a protocol?
A protocol is a document that serves as a work plan for your review that describes the rationale, hypothesis, and planned methods of the review. Your protocol should be prepared before you start your review, even if things change along the way. Following a protocol allows for transparency, reproducibility, and minimizes biases.
For systematic reviews, the PRISMA website provides several sources of guidance on writing a protocol.
For scoping reviews, the Joanna Briggs Institute provides guidance for writing a protocol in section 11.2 of their chapter on scoping reviews. Resources for scoping reviews can be found through the JBI Scoping Review Network.
In general, your protocol should have the following elements:
(Adapted from: Booth, A., Sutton, A. and Papaioannou, D. (2016). Defining the scope. Systematic Approaches to a Successful Literature Review, 2nd edition.)
Registering or Publishing A Completed Protocol
Once you have written your protocol, consider registering it with an organization or publishing it in a journal. Listed below are a few example resources:
PROSPERO - Initiated in early 2011, this international database allows free registration of systematic reviews of interventions and strategies to prevent, diagnose, treat, and monitor health conditions in humans, for which there is a health-related outcome. At the present time, PROSPERO does not accept scoping review protocols.
More information and guidance on registering in PROSPERO can be found on their website.
Now it’s time to execute your search strategy in each of the databases that you have identified. We recommend developing the search strategy in one database before translating the search strategy to the other selected databases. It is also recommended that you create an account with each database in order to save and rerun your searches.
This will make it easier to keep track of things. If you subsequently find terms in the other selected databases, you can then go back and add them to the searches that you have already developed.
Finding studies relevant to your question should not depend solely on database searching: Supplementary search methods, such as grey literature searching, are recommended in order to avoid different forms of bias in what studies are ultimately included in the review.
Grey literature is usually understood to be literature not formally published in books or journals. This can include theses or dissertations, conference proceedings, clinical trials registries, white papers, government reports, and more.
Some grey literature will be retrievable through database searching, but it depends on the databases. Grey literature is also available on websites. One suggestion is to identify associations, organizations, institutions, etc. that are likely to make documents or reports relevant to your question, and then selectively search or browse those sites.
For more information on where to find Grey Literature refer to the Sources Page which can be found within our Systematic and Scoping Review Guide.
To remove duplicate citations there are a few options available. Citation managers, such as Mendeley, Zotero, and EndNote, allow you to remove duplicate citations and assist with citing sources. Also available is systematic review software that will remove duplicates and help with the screening process. Western Libraries subscribes to Covidence and is available to students, faculty, and staff.
In the initial stage of screening, at least two reviewers from the review team will independently scan titles and abstracts of articles that were retrieved from a comprehensive (i.e. multiple source) search, and make decisions whether to include or exclude articles. To do this in a streamlined, unbiased, and method-driven way, reviewers should adhere to the pre-defined eligibility criteria, or guidance form.
Full-Text articles are available within the database and library catalogue. If you cannot find the full-text article you can request a copy through Interlibrary Loans.
The second level of screening is a more rigourous, in-depth process in which the articles that were included in the initial stage of screening are read in full-text. Similar to the initial screening, this is done independently by at least two reviewers from the review team, and the eligibility criteria that was used as a guideline for the initial screening is largely the same.
However, the full-text screening differs in these important ways:
The reason(s) for exclusion must be recorded and reported
You can now screen for outcome(s). Ask yourself: does the study report on the outcome(s) you're interested in?
Although the eligibility criteria is the same, it will require additional detail (clarifying questions may arise during the first stage of screening)
After selecting the full-text articles you can use citation tracking/chaining to ensure that you did not miss any relevant citations in your primary search.
Using the full-text of each article identified for inclusion in the review, extract the pertinent data using a standardized data extraction/coding form. The data extraction form should be as long or as short as necessary and can be coded for computer analysis if desired.
Data synthesis is a process of bringing together data from a set of included studies with the aim of drawing conclusions about a body of evidence. This will include synthesis of study characteristics and, potentially, statistical synthesis of study findings.
For more information on the data synthesis process please refer to Chapter 9 of the Cochrane Handbook.
It can be tempting to jump prematurely into a statistical analysis when undertaking a systematic review. Results of meta-analyses can be very misleading if suitable attention has not been given to formulating the review question; specifying eligibility criteria; identifying and selecting studies; collecting appropriate data; considering risk of bias; planning intervention comparisons; and deciding what data would be meaningful to analyze. Review authors should consult the chapters that precede this one before a meta-analysis is undertaken.
For more information on the data analysis process please refer to Chapter 10 of the Cochrane Handbook.
Conducting a systematic review is a lengthy process and most likely a considerable amount of time has passed since running your primary search. It is recommended that you re-run your primary search in order to ensure that you aren’t missing any articles published since that time.
If you have identified new articles make sure you go back to the Title and Abstract Screening stage and go through each step with this updated set of citations.
Congratulations, you are almost done!
There are different guidelines depending on the review you are undertaking, your review topic, and where you are intending to publish.
Be sure to check with the journal’s guidelines, as well as PRISMA’s Guideline Checklist, to make sure you are including all the necessary elements in your manuscript.
Writing up your review should be an ongoing process throughout the different stages of the systematic review.
