Western Libraries

Systematic & Scoping Reviews

Is a Systematic/Scoping Review Right for You?

Systematic Reviews and Scoping Reviews are types of literature reviews that use systematic methods to collect studies and each have their own complex steps.  ‘New’ researchers will experience a steep learning curve and should be prepared to be trained in the art of searching before the review is conducted. 

To help you understand the difference between a systematic or scoping review as well as provide you with a high-level overview of the work involved, please review the chart below.

  Systematic Review Scoping Review
Definition

A systematic review is a type of literature review that uses systematic methods to collect secondary data, critically appraise research studies, and synthesize findings qualitatively or quantitatively to answer a specific research question.

A scoping review differs from a systematic review in its purpose and aim. The purpose of a scoping review is to provide an overview of the available research evidence without producing a summary answer to a discrete research question.

Average Time to Complete Review 6 months to 1.5 years 9 months to 1 year
Minimum Number of People Require to Complete Review 2 2
Steps for Writing Your Review
  1. Formulate a specific question
  2. Develop a protocol
  3. Conduct search
  4. Select studies and assess study quality
  5. Extract data and analyze/summarize and synthesize relevant studies
  6. Interpret results
  1. Formulate a question
  2. Develop a protocol
  3. Conduct search
  4. Select studies 
  5. Summarize and synthesize relevant studies
  6. Report results and identify gaps and/or trends

If performing a systematic review, we highly recommend that you complete all of the Cochrane Interactive Learning Modules before you begin. 

Should you require support anytime throughout your research process and would like to book a research consult, please contact a member of the Western Libraries Systematic and Scoping Review Team.

If you determine a systematic or scoping review is NOT currently the right path for you and you need research support, please fill out this research consultation form.
 

Average Systematic / Scoping Review Timeline

Month* Task Description Stage
1 - 2 Formulate Research Question Decide on the research question for the review Preparation
1 - 2 Investigate For Previous SRs Search published and unpublished studies (SRs) that answer the same question Preparation
1 - 2 Devise Search Strategy Decide on databases, keywords, and subject headings to find all relevant trials Preparation
2 - 3 Write/Register Protocol Provide an objective, reproducible, sound methodology for peer review Preparation
3 - 5 Primary Search Aim to find all relevant citations even if many irrelevant ones are included Retrieval
3 - 5  Grey Literature Search If applicable - Review of not formally published articles (conference proceedings, government publications, etc.) Retrieval
4 - 5 De-Duplicate Remove identical citations Retrieval
5 - 8 Title & Abstract Screening Screen titles and abstracts, remove irrelevant articles based on inclusion/exclusion criteria Appraisal
5 - 8  Obtain Full Text Download, request copies from authors, inter-library loans, etc. Retrieval
5 - 8  Full Text Screening Exclude irrelevant articles based on inclusion/exclusion criteria Appraisal
9 Supplemental Search Follow citations from included articles to potentially locate additional articles. New articles must go through screening process Retrieval
9 - 11  Data Extraction Obtain the necessary information about study characteristics and findings from the included studies. Synthesis
9 - 11  Data Synthesis Convert extracted data to common representations (usually average and SD) Synthesis
9 - 11  Data Analysis Using qualitative or quantitative data, combine results from all included trials Synthesis
11  Re-Run Primary Search Repeat the search to find new literature published since initial search. New articles must go through screening process Retrieval
1 - 12* Write Up Review Produce and publish final report. Some elements should be written throughout different stages of the review Write-up
* This timeline is only a guide. Systematic Reviews can take up to 2 years depending on the complexity of your topic and the time and resources available to your team

The information below has been adapted from: Cochrane’s Handbook for Systematic Reviews of Interventions & McGill Library’s Systematic Reviews, Scoping Reviews, and other Knowledge Syntheses Guide & UofT Libraries’ Knowledge syntheses: Systematic & Scoping Reviews, and other review types Guide 

 

Formulate Research Question

Formulating a well-constructed research question is essential for a successful review. You should have a draft research question before you choose the type of review that you will conduct to ensure a systematic or scoping review is the appropriate choice.  

A clearly defined question will allow you to focus your search so that it is more efficient and effective, make it easier to find and combine appropriate terms, and help identify relevant results.  

To define and focus your question you can utilize a concept mapping method. Please see our How to Develop a Research Question video as an introductory resource on developing research questions.

 Back To Top

 

Investigate for Previous SRs

Once you have a clearly defined research question, it is important to make sure your question has not already been recently and successfully undertaken. This means it is important to find out if there are other reviews that have been published or that are in the process of being published on your topic. 

Even if you do find another review or synthesis on your topic, it may be sufficiently out of date or you may find other defendable reasons to perform it again. In addition, looking at other reviews published around your topic may also help you refocus your question or redirect your research toward other gaps in the literature. 

To find published reviews or protocols on your topic, you can check resources such as: 

PROSPERO - to find protocols 

Cochrane Library – to find systematic reviews 

Other databases or journals that index systematic/scoping reviews or protocols 

Back To Top

 

Devise Search Strategy

When devising a search strategy, remember it is supposed to be thorough, objective and reproducible. It involves searching more than one database and using a combination of keywords as well as subject headings.   

It is recommended to consult a librarian at this stage of your review. 

For more information on devising a comprehensive search strategy visit our Searching Techniques Page which can be found within our Systematic and Scoping Review Guide. 

Back To Top

 

Write/Register Protocol

What is a protocol? 

A protocol is a document that serves as a work plan for your review that describes the rationale, hypothesis, and planned methods of the review.  Your protocol should be prepared before you start your review, even if things change along the way.  Following a protocol allows for transparency, reproducibility, and minimizes biases. 

For systematic reviews, the PRISMA website provides several sources of guidance on writing a protocol

For scoping reviews, the Joanna Briggs Institute provides guidance for writing a protocol in section 11.2 of their chapter on scoping reviews

In general, your protocol should have the following elements: 

  • Background literature review 
  • Review question 
  • Criteria for inclusion/exclusion of studies 
  • Types of studies, populations, interventions/exposures, outcome measures 
  • Search strategy for identification of studies 
  • Study selection methods 
  • Assessment of methodological quality (if applicable) 
  • Data extraction and synthesis 
  • Timeframe for conducting the review 

(Adapted from: Booth, A., Sutton, A. and Papaioannou, D. (2016). Defining the scope. Systematic Approaches to a Successful Literature Review, 2nd edition.) 

 

Registering or Publishing A Completed Protocol

Once you have written your protocol, consider registering it with an organization or publishing it in a journal.  Listed below are a few example resources: 

  • PROSPERO - Initiated in early 2011, this international database allows free registration of systematic reviews of interventions and strategies to prevent, diagnose, treat, and monitor health conditions in humans, for which there is a health-related outcome. At the present time, PROSPERO does not accept scoping review protocols. 

More information and guidance on registering in PROSPERO can be found on their website

  • OSF Registries - Use the OSF (Open Science Framework) platform to preregister the protocol for your knowledge synthesis. This is a useful option if you are not publishing a systematic review or a review of interventions with health-related outcomes. 
  • BioMed Central Protocols - BioMed Central will consider protocols of any type of research for publication, following the standard peer review. 
  • BMJ Open - BMJ Open "will consider publishing without peer review protocols that have formal ethical approval and funding from a recognized, open access advocating research-funding body". Otherwise, protocols are peer reviewed. 
  • Systematic Reviews, a BioMed Central journal - This open access title publishes protocols of systematic reviews broadly related to health sciences. 

Back To Top

 

Primary Search

Now it’s time to execute your search strategy in each of the databases that you have identified.  We recommend developing the search strategy in one database before translating the search strategy to the other selected databases. It is also recommended that you create an account with each database in order to save and rerun your searches. 

This will make it easier to keep track of things. If you subsequently find terms in the other selected databases, you can then go back and add them to the searches that you have already developed. 

Back To Top

 

Grey Literature Search

Finding studies relevant to your question should not depend solely on database searching: Supplementary search methods, such as grey literature searching, are recommended in order to avoid different forms of bias in what studies are ultimately included in the review. 

Grey literature is usually understood to be literature not formally published in books or journals. This can include theses or dissertations, conference proceedings, clinical trials registries, white papers, government reports, and more. 

Some grey literature will be retrievable through database searching, but it depends on the databases.  Grey literature is also available on websites. One suggestion is to identify associations, organizations, institutions, etc. that are likely to make documents or reports relevant to your question, and then selectively search or browse those sites. 

For more information on where to find Grey Literature refer to the Sources Page which can be found within our Systematic and Scoping Review Guide. 

Back To Top

 

De-Duplicate

To remove duplicate citations there are a few options available. Citation managers, such as Mendeley, Zotero, and EndNote, allow you to remove duplicate citations and assist with citing sources. Also available is systematic review software that will remove duplicates and help with the screening process. Western Libraries subscribes to Covidence and is available to students, faculty, and staff.  

Back To Top

 

Title and Abstract Screening

In the initial stage of screening, at least two reviewers from the review team will independently scan titles and abstracts of articles that were retrieved from a comprehensive (i.e. multiple source) search, and make decisions whether to include or exclude articles. To do this in a streamlined, unbiased, and method-driven way, reviewers should adhere to the pre-defined eligibility criteria, or guidance form. 

Back To Top

 

Obtain Full Text

Full-Text articles are available within the database and library catalogue. If you cannot find the full-text article you can request a copy through RACER, our inter-library loan service.  

Back To Top

 

Full Text Screening

The second level of screening is a more rigourous, in-depth process in which the articles that were included in the initial stage of screening are read in full-text. Similar to the initial screening, this is done independently by at least two reviewers from the review team, and the eligibility criteria that was used as a guideline for the initial screening is largely the same. 

However, the full-text screening differs in these important ways: 

  • The reason(s) for exclusion must be recorded and reported 

  • You can now screen for outcome(s). Ask yourself: does the study report on the outcome(s) you're interested in? 

  • Although the eligibility criteria is the same, it will require additional detail (clarifying questions may arise during the first stage of screening) 

Back To Top

 

Supplemental Search

After selecting the full-text articles you can use citation tracking/chaining to ensure that you did not miss any relevant citations in your primary search.  

Back To Top

 

Data Extraction

Using the full-text of each article identified for inclusion in the review, extract the pertinent data using a standardized data extraction/coding form. The data extraction form should be as long or as short as necessary and can be coded for computer analysis if desired. 

Back To Top

 

Data Synthesis

Data synthesis is a process of bringing together data from a set of included studies with the aim of drawing conclusions about a body of evidence. This will include synthesis of study characteristics and, potentially, statistical synthesis of study findings. 

For more information on the data synthesis process please refer to Chapter 9 of the Cochrane Handbook

Back To Top

 

Data Analysis

It can be tempting to jump prematurely into a statistical analysis when undertaking a systematic review. Results of meta-analyses can be very misleading if suitable attention has not been given to formulating the review question; specifying eligibility criteria; identifying and selecting studies; collecting appropriate data; considering risk of bias; planning intervention comparisons; and deciding what data would be meaningful to analyze. Review authors should consult the chapters that precede this one before a meta-analysis is undertaken. 

For more information on the data analysis process please refer to Chapter 10 of the Cochrane Handbook.   

Back To Top

 

Re-Run Primary Search

Conducting a systematic review is a lengthy process and most likely a considerable amount of time has passed since running your primary search. It is recommended that you re-run your primary search in order to ensure that you aren’t missing any articles published since that time. 
If you have identified new articles make sure you go back to the Title and Abstract Screening stage and go through each step with this updated set of citations.  

Back To Top

 

Write-Up Review

Congratulations, you are almost done!   

There are different guidelines depending on the review you are undertaking, your review topic, and where you are intending to publish. 
Be sure to check with the journal’s guidelines, as well as PRISMA’s Guideline Checklist, to make sure you are including all the necessary elements in your manuscript. 

Writing up your review should be an ongoing process throughout the different stages of the systematic review. 

Back To Top