18. Risk of bias in studies
Present assessments of risk of bias for each included study
Essential elements
- Present tables or figures indicating for each study the risk of bias in each domain/component/item assessed and overall study-level risk of bias.
- Present justification for each risk of bias judgment—for example, in the form of relevant quotations from reports of included studies.
Additional elements
- If assessments of risk of bias were done for specific outcomes or results in each study, consider displaying risk of bias judgments on a forest plot, next to the study results, so that the limitations of studies contributing to a particular meta-analysis are evident (see Sterne et al1 for an example forest plot).
Explanation
For readers to understand the internal validity of a systematic review’s results, they need to know the risk of bias in results of each included study. Reporting only summary data (such as “two of eight studies successfully blinded participants”) is inadequate because it fails to inform readers which studies had each particular methodological shortcoming. A more informative approach is to present tables or figures indicating for each study the risk of bias in each domain/component/item assessed (such as blinding of outcome assessors, missing outcome data), so that users can understand what factors led to the overall study-level risk of bias judgment.23
Example
“We used the RoB 2.0 tool to assess risk of bias for each of the included studies. A summary of these assessments is provided in Table 1. In terms of overall risk of bias, there were concerns about risk of bias for the majority of studies (20/24), with two of these assessed as at high risk of bias (Musher‐Eizenman 2010; Wansink 2013a). A text summary is provided below for each of the six individual components of the ‘Risk of bias’ assessment. Justifications for assessments are available at the following (https://dx.doi.org/10.6084/m9.figshare.9159824).”4
Table 1: The table displays for each included study the risk-of-bias judgment for each of six domains of bias, and for the overall risk of bias in two results (selection of a product, consumption of a product); the following is an abridged version of the table presented in the review. Reproduced from Hollands et al.4
Study Bias arising from the randomisation process Bias arising from the timing of identification and recruitment of individual participants in relation to timing of randomisation (CRCT only) Bias due to deviations from intended interventions Bias due to missing outcome data Bias in measurement of the outcome Bias in selection of the reported result Overall risk of bias (selection of a product) Overall risk of bias (consumption of a product) Fiske 2004 Some concerns Low risk Low risk Low risk Low risk Low risk Some concerns Not applicable Foster 2014 Low risk Low risk Low risk Low risk Low risk Low risk Low risk Not applicable Kocken 2012 Some concerns Low risk Low risk Low risk Low risk Low risk Some concerns Not applicable Pechey 2019 Some concerns Not applicable Low risk Low risk Low risk Low risk Some concerns Not applicable Roe 2013 Some concerns Not applicable Low risk Low risk Low risk Low risk Some concerns Some concerns Stubbs 2001 Some concerns Not applicable Low risk Low risk Low risk Low risk Not applicable Some concerns CRCT: cluster-randomised controlled trials. Justifications for assessments are available at the following (https://dx.doi.org/10.6084/m9.figshare.9159824).
Training
The UK EQUATOR Centre runs training on how to write using reporting guidelines.
Discuss this item
Visit this items’ discussion page to ask questions and give feedback.
