TY - JOUR
T1 - GRADE approach to rate the certainty from a network meta-analysis
T2 - avoiding spurious judgments of imprecision in sparse networks
AU - GRADE Working Group
AU - Brignardello-Petersen, Romina
AU - Murad, M. Hassan
AU - Walter, Stephen D.
AU - McLeod, Shelley
AU - Carrasco-Labra, Alonso
AU - Rochwerg, Bram
AU - Schünemann, Holger J.
AU - Tomlinson, George
AU - Guyatt, Gordon H.
N1 - Publisher Copyright:
© 2018 Elsevier Inc.
PY - 2019/1
Y1 - 2019/1
N2 - When direct and indirect estimates of treatment effects are coherent, network meta-analysis (NMA) estimates should have increased precision (narrower confidence or credible intervals compared with relying on direct estimates alone), a benefit of NMA. We have, however, observed cases of sparse networks in which combining direct and indirect estimates results in marked widening of the confidence intervals. In many cases, the assumption of common between-study heterogeneity across the network seems to be responsible for this counterintuitive result. Although the assumption of common between-study heterogeneity across paired comparisons may, in many cases, not be appropriate, it is required to ensure the feasibility of estimating NMA treatment effects. This is especially the case in sparse networks, in which data are insufficient to reliably estimate different variances across the network. The result, however, may be spuriously wide confidence intervals for some of the comparisons in the network (and, in the Grading of Recommendations Assessment, Development, and Evaluation approach, inappropriately low ratings of the certainty of the evidence through rating down for serious imprecision). Systematic reviewers should be aware of the problem and plan sensitivity analyses that produce intuitively sensible confidence intervals. These sensitivity analyses may include using informative priors for the between-study heterogeneity parameter in the Bayesian framework and the use of fixed effects models.
AB - When direct and indirect estimates of treatment effects are coherent, network meta-analysis (NMA) estimates should have increased precision (narrower confidence or credible intervals compared with relying on direct estimates alone), a benefit of NMA. We have, however, observed cases of sparse networks in which combining direct and indirect estimates results in marked widening of the confidence intervals. In many cases, the assumption of common between-study heterogeneity across the network seems to be responsible for this counterintuitive result. Although the assumption of common between-study heterogeneity across paired comparisons may, in many cases, not be appropriate, it is required to ensure the feasibility of estimating NMA treatment effects. This is especially the case in sparse networks, in which data are insufficient to reliably estimate different variances across the network. The result, however, may be spuriously wide confidence intervals for some of the comparisons in the network (and, in the Grading of Recommendations Assessment, Development, and Evaluation approach, inappropriately low ratings of the certainty of the evidence through rating down for serious imprecision). Systematic reviewers should be aware of the problem and plan sensitivity analyses that produce intuitively sensible confidence intervals. These sensitivity analyses may include using informative priors for the between-study heterogeneity parameter in the Bayesian framework and the use of fixed effects models.
KW - GRADE
KW - Network meta-analysis
KW - certainty
KW - clinical practice guidelines
KW - evidence-based medicine
KW - imprecision
KW - meta-analysis
KW - quality of evidence
UR - http://www.scopus.com/inward/record.url?scp=85054793352&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85054793352&partnerID=8YFLogxK
U2 - 10.1016/j.jclinepi.2018.08.022
DO - 10.1016/j.jclinepi.2018.08.022
M3 - Article
C2 - 30253217
AN - SCOPUS:85054793352
SN - 0895-4356
VL - 105
SP - 60
EP - 67
JO - Journal of Clinical Epidemiology
JF - Journal of Clinical Epidemiology
ER -