GRADE guidelines: 4. Rating the quality of evidence - Study limitations (risk of bias)

Gordon H. Guyatt, Andrew D. Oxman, Gunn Vist, Regina Kunz, Jan Brozek, Pablo Alonso-Coello, Victor Manuel Montori, Elie A. Akl, Ben Djulbegovic, Yngve Falck-Ytter, Susan L. Norris, John W. Williams, David Atkins, Joerg Meerpohl, Holger J. Schünemann

Research output: Contribution to journalArticle

1046 Citations (Scopus)

Abstract

In the GRADE approach, randomized trials start as high-quality evidence and observational studies as low-quality evidence, but both can be rated down if most of the relevant evidence comes from studies that suffer from a high risk of bias. Well-established limitations of randomized trials include failure to conceal allocation, failure to blind, loss to follow-up, and failure to appropriately consider the intention-to-treat principle. More recently recognized limitations include stopping early for apparent benefit and selective reporting of outcomes according to the results. Key limitations of observational studies include use of inappropriate controls and failure to adequately adjust for prognostic imbalance. Risk of bias may vary across outcomes (e.g., loss to follow-up may be far less for all-cause mortality than for quality of life), a consideration that many systematic reviews ignore. In deciding whether to rate down for risk of bias - whether for randomized trials or observational studies - authors should not take an approach that averages across studies. Rather, for any individual outcome, when there are some studies with a high risk, and some with a low risk of bias, they should consider including only the studies with a lower risk of bias.

Original languageEnglish (US)
Pages (from-to)407-415
Number of pages9
JournalJournal of Clinical Epidemiology
Volume64
Issue number4
DOIs
StatePublished - Apr 2011

Fingerprint

Guidelines
Observational Studies
Quality of Life
Mortality

Keywords

  • blinding
  • concealment
  • confidence in estimates
  • GRADE
  • quality of evidence
  • risk of bias

ASJC Scopus subject areas

  • Epidemiology

Cite this

Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., ... Schünemann, H. J. (2011). GRADE guidelines: 4. Rating the quality of evidence - Study limitations (risk of bias). Journal of Clinical Epidemiology, 64(4), 407-415. https://doi.org/10.1016/j.jclinepi.2010.07.017

GRADE guidelines : 4. Rating the quality of evidence - Study limitations (risk of bias). / Guyatt, Gordon H.; Oxman, Andrew D.; Vist, Gunn; Kunz, Regina; Brozek, Jan; Alonso-Coello, Pablo; Montori, Victor Manuel; Akl, Elie A.; Djulbegovic, Ben; Falck-Ytter, Yngve; Norris, Susan L.; Williams, John W.; Atkins, David; Meerpohl, Joerg; Schünemann, Holger J.

In: Journal of Clinical Epidemiology, Vol. 64, No. 4, 04.2011, p. 407-415.

Research output: Contribution to journalArticle

Guyatt, GH, Oxman, AD, Vist, G, Kunz, R, Brozek, J, Alonso-Coello, P, Montori, VM, Akl, EA, Djulbegovic, B, Falck-Ytter, Y, Norris, SL, Williams, JW, Atkins, D, Meerpohl, J & Schünemann, HJ 2011, 'GRADE guidelines: 4. Rating the quality of evidence - Study limitations (risk of bias)', Journal of Clinical Epidemiology, vol. 64, no. 4, pp. 407-415. https://doi.org/10.1016/j.jclinepi.2010.07.017
Guyatt, Gordon H. ; Oxman, Andrew D. ; Vist, Gunn ; Kunz, Regina ; Brozek, Jan ; Alonso-Coello, Pablo ; Montori, Victor Manuel ; Akl, Elie A. ; Djulbegovic, Ben ; Falck-Ytter, Yngve ; Norris, Susan L. ; Williams, John W. ; Atkins, David ; Meerpohl, Joerg ; Schünemann, Holger J. / GRADE guidelines : 4. Rating the quality of evidence - Study limitations (risk of bias). In: Journal of Clinical Epidemiology. 2011 ; Vol. 64, No. 4. pp. 407-415.
@article{8e8312552121460eb28db4bd38d0e347,
title = "GRADE guidelines: 4. Rating the quality of evidence - Study limitations (risk of bias)",
abstract = "In the GRADE approach, randomized trials start as high-quality evidence and observational studies as low-quality evidence, but both can be rated down if most of the relevant evidence comes from studies that suffer from a high risk of bias. Well-established limitations of randomized trials include failure to conceal allocation, failure to blind, loss to follow-up, and failure to appropriately consider the intention-to-treat principle. More recently recognized limitations include stopping early for apparent benefit and selective reporting of outcomes according to the results. Key limitations of observational studies include use of inappropriate controls and failure to adequately adjust for prognostic imbalance. Risk of bias may vary across outcomes (e.g., loss to follow-up may be far less for all-cause mortality than for quality of life), a consideration that many systematic reviews ignore. In deciding whether to rate down for risk of bias - whether for randomized trials or observational studies - authors should not take an approach that averages across studies. Rather, for any individual outcome, when there are some studies with a high risk, and some with a low risk of bias, they should consider including only the studies with a lower risk of bias.",
keywords = "blinding, concealment, confidence in estimates, GRADE, quality of evidence, risk of bias",
author = "Guyatt, {Gordon H.} and Oxman, {Andrew D.} and Gunn Vist and Regina Kunz and Jan Brozek and Pablo Alonso-Coello and Montori, {Victor Manuel} and Akl, {Elie A.} and Ben Djulbegovic and Yngve Falck-Ytter and Norris, {Susan L.} and Williams, {John W.} and David Atkins and Joerg Meerpohl and Sch{\"u}nemann, {Holger J.}",
year = "2011",
month = "4",
doi = "10.1016/j.jclinepi.2010.07.017",
language = "English (US)",
volume = "64",
pages = "407--415",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",
number = "4",

}

TY - JOUR

T1 - GRADE guidelines

T2 - 4. Rating the quality of evidence - Study limitations (risk of bias)

AU - Guyatt, Gordon H.

AU - Oxman, Andrew D.

AU - Vist, Gunn

AU - Kunz, Regina

AU - Brozek, Jan

AU - Alonso-Coello, Pablo

AU - Montori, Victor Manuel

AU - Akl, Elie A.

AU - Djulbegovic, Ben

AU - Falck-Ytter, Yngve

AU - Norris, Susan L.

AU - Williams, John W.

AU - Atkins, David

AU - Meerpohl, Joerg

AU - Schünemann, Holger J.

PY - 2011/4

Y1 - 2011/4

N2 - In the GRADE approach, randomized trials start as high-quality evidence and observational studies as low-quality evidence, but both can be rated down if most of the relevant evidence comes from studies that suffer from a high risk of bias. Well-established limitations of randomized trials include failure to conceal allocation, failure to blind, loss to follow-up, and failure to appropriately consider the intention-to-treat principle. More recently recognized limitations include stopping early for apparent benefit and selective reporting of outcomes according to the results. Key limitations of observational studies include use of inappropriate controls and failure to adequately adjust for prognostic imbalance. Risk of bias may vary across outcomes (e.g., loss to follow-up may be far less for all-cause mortality than for quality of life), a consideration that many systematic reviews ignore. In deciding whether to rate down for risk of bias - whether for randomized trials or observational studies - authors should not take an approach that averages across studies. Rather, for any individual outcome, when there are some studies with a high risk, and some with a low risk of bias, they should consider including only the studies with a lower risk of bias.

AB - In the GRADE approach, randomized trials start as high-quality evidence and observational studies as low-quality evidence, but both can be rated down if most of the relevant evidence comes from studies that suffer from a high risk of bias. Well-established limitations of randomized trials include failure to conceal allocation, failure to blind, loss to follow-up, and failure to appropriately consider the intention-to-treat principle. More recently recognized limitations include stopping early for apparent benefit and selective reporting of outcomes according to the results. Key limitations of observational studies include use of inappropriate controls and failure to adequately adjust for prognostic imbalance. Risk of bias may vary across outcomes (e.g., loss to follow-up may be far less for all-cause mortality than for quality of life), a consideration that many systematic reviews ignore. In deciding whether to rate down for risk of bias - whether for randomized trials or observational studies - authors should not take an approach that averages across studies. Rather, for any individual outcome, when there are some studies with a high risk, and some with a low risk of bias, they should consider including only the studies with a lower risk of bias.

KW - blinding

KW - concealment

KW - confidence in estimates

KW - GRADE

KW - quality of evidence

KW - risk of bias

UR - http://www.scopus.com/inward/record.url?scp=79951955368&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79951955368&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2010.07.017

DO - 10.1016/j.jclinepi.2010.07.017

M3 - Article

C2 - 21247734

AN - SCOPUS:79951955368

VL - 64

SP - 407

EP - 415

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

IS - 4

ER -