UpToDate adherence to GRADE criteria for strong recommendations: an analytical survey

Thomas Agoritsas, Arnaud Merglen, Anja Fog Heen, Annette Kristiansen, Ignacio Neumann, Juan Brito Campana, Romina Brignardello-Petersen, Paul E. Alexander, David M. Rind, Per O. Vandvik, Gordon H. Guyatt

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

INTRODUCTION: UpToDate is widely used by clinicians worldwide and includes more than 9400 recommendations that apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. GRADE guidance warns against strong recommendations when certainty of the evidence is low or very low (discordant recommendations) but has identified five paradigmatic situations in which discordant recommendations may be justified.

OBJECTIVES: Our objective was to document the strength of recommendations in UpToDate and assess the frequency and appropriateness of discordant recommendations.

DESIGN: Analytical survey of all recommendations in UpToDate.

METHODS: We identified all GRADE recommendations in UpToDate and examined their strength (strong or weak) and certainty of the evidence (high, moderate or low certainty). We identified all discordant recommendations as of January 2015, and pairs of reviewers independently classified them either into one of the five appropriate paradigms or into one of three categories inconsistent with GRADE guidance, based on the evidence presented in UpToDate.

RESULTS: UpToDate included 9451 GRADE recommendations, of which 6501 (68.8%) were formulated as weak recommendations and 2950 (31.2%) as strong. Among the strong, 844 (28.6%) were based on high certainty in effect estimates, 1740 (59.0%) on moderate certainty and 366 (12.4%) on low certainty. Of the 349 discordant recommendations 204 (58.5%) were judged appropriately (consistent with one of the five paradigms); we classified 47 (13.5%) as good practice statements; 38 (10.9%) misclassified the evidence as low certainty when it was at least moderate and 60 (17.2%) warranted a weak rather than a strong recommendation.

CONCLUSION: The proportion of discordant recommendations in UpToDate is small (3.7% of all recommendations) and the proportion that is truly problematic (strong recommendations that would best have been weak) is very small (0.6%). Clinicians should nevertheless be cautious and look for clear explanations-in UpToDate and elsewhere-when guidelines offer strong recommendations based on low certainty evidence.

Original languageEnglish (US)
Pages (from-to)e018593
JournalBMJ Open
Volume7
Issue number11
DOIs
StatePublished - Nov 16 2017

Fingerprint

Surveys and Questionnaires
Guidelines

Keywords

  • clinical decision making
  • clinical practice guidelines
  • evidence-based medicine
  • quality of the evidence
  • strength of recommendations

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Agoritsas, T., Merglen, A., Heen, A. F., Kristiansen, A., Neumann, I., Brito Campana, J., ... Guyatt, G. H. (2017). UpToDate adherence to GRADE criteria for strong recommendations: an analytical survey. BMJ Open, 7(11), e018593. https://doi.org/10.1136/bmjopen-2017-018593

UpToDate adherence to GRADE criteria for strong recommendations : an analytical survey. / Agoritsas, Thomas; Merglen, Arnaud; Heen, Anja Fog; Kristiansen, Annette; Neumann, Ignacio; Brito Campana, Juan; Brignardello-Petersen, Romina; Alexander, Paul E.; Rind, David M.; Vandvik, Per O.; Guyatt, Gordon H.

In: BMJ Open, Vol. 7, No. 11, 16.11.2017, p. e018593.

Research output: Contribution to journalArticle

Agoritsas, T, Merglen, A, Heen, AF, Kristiansen, A, Neumann, I, Brito Campana, J, Brignardello-Petersen, R, Alexander, PE, Rind, DM, Vandvik, PO & Guyatt, GH 2017, 'UpToDate adherence to GRADE criteria for strong recommendations: an analytical survey', BMJ Open, vol. 7, no. 11, pp. e018593. https://doi.org/10.1136/bmjopen-2017-018593
Agoritsas, Thomas ; Merglen, Arnaud ; Heen, Anja Fog ; Kristiansen, Annette ; Neumann, Ignacio ; Brito Campana, Juan ; Brignardello-Petersen, Romina ; Alexander, Paul E. ; Rind, David M. ; Vandvik, Per O. ; Guyatt, Gordon H. / UpToDate adherence to GRADE criteria for strong recommendations : an analytical survey. In: BMJ Open. 2017 ; Vol. 7, No. 11. pp. e018593.
@article{b4cd046e8b5942bba1620366a3e43f90,
title = "UpToDate adherence to GRADE criteria for strong recommendations: an analytical survey",
abstract = "INTRODUCTION: UpToDate is widely used by clinicians worldwide and includes more than 9400 recommendations that apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. GRADE guidance warns against strong recommendations when certainty of the evidence is low or very low (discordant recommendations) but has identified five paradigmatic situations in which discordant recommendations may be justified.OBJECTIVES: Our objective was to document the strength of recommendations in UpToDate and assess the frequency and appropriateness of discordant recommendations.DESIGN: Analytical survey of all recommendations in UpToDate.METHODS: We identified all GRADE recommendations in UpToDate and examined their strength (strong or weak) and certainty of the evidence (high, moderate or low certainty). We identified all discordant recommendations as of January 2015, and pairs of reviewers independently classified them either into one of the five appropriate paradigms or into one of three categories inconsistent with GRADE guidance, based on the evidence presented in UpToDate.RESULTS: UpToDate included 9451 GRADE recommendations, of which 6501 (68.8{\%}) were formulated as weak recommendations and 2950 (31.2{\%}) as strong. Among the strong, 844 (28.6{\%}) were based on high certainty in effect estimates, 1740 (59.0{\%}) on moderate certainty and 366 (12.4{\%}) on low certainty. Of the 349 discordant recommendations 204 (58.5{\%}) were judged appropriately (consistent with one of the five paradigms); we classified 47 (13.5{\%}) as good practice statements; 38 (10.9{\%}) misclassified the evidence as low certainty when it was at least moderate and 60 (17.2{\%}) warranted a weak rather than a strong recommendation.CONCLUSION: The proportion of discordant recommendations in UpToDate is small (3.7{\%} of all recommendations) and the proportion that is truly problematic (strong recommendations that would best have been weak) is very small (0.6{\%}). Clinicians should nevertheless be cautious and look for clear explanations-in UpToDate and elsewhere-when guidelines offer strong recommendations based on low certainty evidence.",
keywords = "clinical decision making, clinical practice guidelines, evidence-based medicine, quality of the evidence, strength of recommendations",
author = "Thomas Agoritsas and Arnaud Merglen and Heen, {Anja Fog} and Annette Kristiansen and Ignacio Neumann and {Brito Campana}, Juan and Romina Brignardello-Petersen and Alexander, {Paul E.} and Rind, {David M.} and Vandvik, {Per O.} and Guyatt, {Gordon H.}",
year = "2017",
month = "11",
day = "16",
doi = "10.1136/bmjopen-2017-018593",
language = "English (US)",
volume = "7",
pages = "e018593",
journal = "BMJ Open",
issn = "2044-6055",
publisher = "BMJ Publishing Group",
number = "11",

}

TY - JOUR

T1 - UpToDate adherence to GRADE criteria for strong recommendations

T2 - an analytical survey

AU - Agoritsas, Thomas

AU - Merglen, Arnaud

AU - Heen, Anja Fog

AU - Kristiansen, Annette

AU - Neumann, Ignacio

AU - Brito Campana, Juan

AU - Brignardello-Petersen, Romina

AU - Alexander, Paul E.

AU - Rind, David M.

AU - Vandvik, Per O.

AU - Guyatt, Gordon H.

PY - 2017/11/16

Y1 - 2017/11/16

N2 - INTRODUCTION: UpToDate is widely used by clinicians worldwide and includes more than 9400 recommendations that apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. GRADE guidance warns against strong recommendations when certainty of the evidence is low or very low (discordant recommendations) but has identified five paradigmatic situations in which discordant recommendations may be justified.OBJECTIVES: Our objective was to document the strength of recommendations in UpToDate and assess the frequency and appropriateness of discordant recommendations.DESIGN: Analytical survey of all recommendations in UpToDate.METHODS: We identified all GRADE recommendations in UpToDate and examined their strength (strong or weak) and certainty of the evidence (high, moderate or low certainty). We identified all discordant recommendations as of January 2015, and pairs of reviewers independently classified them either into one of the five appropriate paradigms or into one of three categories inconsistent with GRADE guidance, based on the evidence presented in UpToDate.RESULTS: UpToDate included 9451 GRADE recommendations, of which 6501 (68.8%) were formulated as weak recommendations and 2950 (31.2%) as strong. Among the strong, 844 (28.6%) were based on high certainty in effect estimates, 1740 (59.0%) on moderate certainty and 366 (12.4%) on low certainty. Of the 349 discordant recommendations 204 (58.5%) were judged appropriately (consistent with one of the five paradigms); we classified 47 (13.5%) as good practice statements; 38 (10.9%) misclassified the evidence as low certainty when it was at least moderate and 60 (17.2%) warranted a weak rather than a strong recommendation.CONCLUSION: The proportion of discordant recommendations in UpToDate is small (3.7% of all recommendations) and the proportion that is truly problematic (strong recommendations that would best have been weak) is very small (0.6%). Clinicians should nevertheless be cautious and look for clear explanations-in UpToDate and elsewhere-when guidelines offer strong recommendations based on low certainty evidence.

AB - INTRODUCTION: UpToDate is widely used by clinicians worldwide and includes more than 9400 recommendations that apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. GRADE guidance warns against strong recommendations when certainty of the evidence is low or very low (discordant recommendations) but has identified five paradigmatic situations in which discordant recommendations may be justified.OBJECTIVES: Our objective was to document the strength of recommendations in UpToDate and assess the frequency and appropriateness of discordant recommendations.DESIGN: Analytical survey of all recommendations in UpToDate.METHODS: We identified all GRADE recommendations in UpToDate and examined their strength (strong or weak) and certainty of the evidence (high, moderate or low certainty). We identified all discordant recommendations as of January 2015, and pairs of reviewers independently classified them either into one of the five appropriate paradigms or into one of three categories inconsistent with GRADE guidance, based on the evidence presented in UpToDate.RESULTS: UpToDate included 9451 GRADE recommendations, of which 6501 (68.8%) were formulated as weak recommendations and 2950 (31.2%) as strong. Among the strong, 844 (28.6%) were based on high certainty in effect estimates, 1740 (59.0%) on moderate certainty and 366 (12.4%) on low certainty. Of the 349 discordant recommendations 204 (58.5%) were judged appropriately (consistent with one of the five paradigms); we classified 47 (13.5%) as good practice statements; 38 (10.9%) misclassified the evidence as low certainty when it was at least moderate and 60 (17.2%) warranted a weak rather than a strong recommendation.CONCLUSION: The proportion of discordant recommendations in UpToDate is small (3.7% of all recommendations) and the proportion that is truly problematic (strong recommendations that would best have been weak) is very small (0.6%). Clinicians should nevertheless be cautious and look for clear explanations-in UpToDate and elsewhere-when guidelines offer strong recommendations based on low certainty evidence.

KW - clinical decision making

KW - clinical practice guidelines

KW - evidence-based medicine

KW - quality of the evidence

KW - strength of recommendations

UR - http://www.scopus.com/inward/record.url?scp=85049407148&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85049407148&partnerID=8YFLogxK

U2 - 10.1136/bmjopen-2017-018593

DO - 10.1136/bmjopen-2017-018593

M3 - Article

C2 - 29150475

AN - SCOPUS:85049407148

VL - 7

SP - e018593

JO - BMJ Open

JF - BMJ Open

SN - 2044-6055

IS - 11

ER -