Background: Systematic reviews should assess risk of bias in included studies, in order to draw conclusions about the strength of the evidence for causal effects of interventions on health outcomes. The ROBINS-I (Risk Of Bias In Non-randomised Studies - of Interventions) tool provides a structured approach to assessing risk of bias in non-randomised studies of interventions (NRSI) based on comparisons with a hypothetical high-quality pragmatic randomised trial. Answers to signalling questions lead to judgements of risk of bias within bias domains and overall. The published tool and guidance mainly focus on studies with a cohort-type design. Instrumental variable (IV) analyses, which can estimate causal effects of interventions in the presence of unmeasured confounding of the intervention-outcome association but require other assumptions, are not currently addressed.
Objectives: Adapt the ROBINS-I tool, including its signalling questions and accompanying guidance, to assess risk of bias in studies reporting IV analyses.
Methods: An international working group of experts in IV methodology met via teleconferences and face-to-face. The group agreed changes to the ROBINS-I tool through consensus.
Results: We added two new bias domains to ROBINS-I. The first (replacing the ROBINS-I confounding domain) assesses the core IV assumptions that 1) the IV is associated with the intervention; 2) there is no residual confounding of the IV-outcome association; and, 3) the effect of the IV on the outcome is via the intervention. The second assesses the plausibility of the additional ‘point identifying’ assumptions required for the validity of different types of IV estimate. In addition we adapted signalling questions within existing ROBINS-I bias domains, such as the ‘deviations from intended intervention’ domain.
Conclusions: The extended ROBINS-I tool contains signalling questions and guidance to assess risk of bias in studies reporting IV analyses. These will now be piloted, with feedback used to inform further modifications.