Risk of Bias in Systematic Reviews
Evidence syntheses strive to eliminate bias in their findings. The risk of bias assessment (sometimes called "quality assessment" or "critical appraisal") helps to establish transparency of evidence synthesis results and findings. A risk of bias assessment is often performed for each included study in your review. Individual studies that are included in a synthesis may include biases in their results or conclusions (for example, design flaws that raise questions about the validity of their findings or an overestimate of an intervention's effect).
A risk of bias assessment is generally not required for evidence synthesis methods outside of systematic reviews. However, guidelines and recommendations for evidence synthesis methods are continually evolving. Be sure to check on the most recent guidance for the method you are utilizing.
Scoping reviews don't typically include a risk of bias assessment.
“A key difference between scoping reviews and systematic reviews is that the former are generally conducted to provide an overview of the existing evidence regardless of methodological quality or risk of bias.... Therefore, the included sources of evidence are typically not critically appraised for scoping reviews.”
From Tricco et al. (2018). PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467–473. See the full article for more methodology guidelines specific to scoping reviews.
The Cochrane Training presentation linked below helps to navigate the steps of risk of bias assessment, but it discusses health studies-based sources of bias that might not apply to other disciplines.
For disciplines outside of human medicine, the Critical Appraisal Skills Programme provides checklists that can be applied to a diverse array of study types, and the Mixed Methods Appraisal Tool can be a helpful resource if the included studies use mixed methods. See the links for these below as well.
The results of your risk of bias assessment can be represented in a table format in your evidence synthesis. The table should list each included study and show how strong it is across several quality criteria for that particular study type.
If a high proportion of your included studies are assessed with a high risk of bias, caution should be used when interpreting the results for your evidence synthesis. More information can be found in Chapters 7 and 8 of the Cochrane Handbook:
This is an example of a table presenting the results of a risk of bias assessment. Red: high risk; yellow: unclear risk; green: low risk.
Source: Liu, Z., et al. (2015). Bed rest versus early ambulation with standard anticoagulation in the management of deep vein thrombosis: A meta-analysis. PloS One, 10(4), Article e0121388. Licensed under CC BY 4.0.