Above the GRADE

Evaluation of Guidelines in Critical Care Medicine

Charles R. Sims, Matthew Warner, Henry Thomas Stelfox, Joseph A. Hyder

Research output: Contribution to journalReview article

1 Citation (Scopus)

Abstract

Objectives: We examined recommendations within critical care guidelines to describe the pairing patterns for strength of recommendation and quality of evidence. We further identified recommendations where the reported strength of recommendation was strong while the reported quality of evidence was not high/moderate and then assessed whether such pairings were within five paradigmatic situations offered by Grading of Recommendations Assessment, Development and Evaluation methodology to justify such pairings. Data Sources and Extraction: We identified all clinical critical care guidelines published online from 2011 to 2017 by the Society of Critical Care Medicine along with individual guidelines published by Surviving Sepsis Campaign, Kidney Disease Improving Global Outcomes, American Society for Parenteral and Enteral Nutrition, and the Infectious Disease Society of America/American Thoracic Society. Data Synthesis: In all, 15 documents specifying 681 eligible recommendations demonstrated variation in strength of recommendation (strong n = 215 [31.6%], weak n = 345 [50.7%], none n = 121 [17.8%]) and in quality of evidence (high n = 41 [6.0%], moderate n = 151 [22.2%], low/very low n = 298 [43.8%], and Expert Consensus/none n = 191 [28.1%]). Strength of recommendation and quality of evidence were positively correlated (ρ = 0.66; p < 0.0001). Of 215 strong recommendations, 69 (32.1%) were discordantly paired with evidence other than high/moderate. Twenty-two of 69 (31.9%) involved Strong/Expert Consensus recommendations, a category discouraged by Grading of Recommendations Assessment, Development and Evaluation methodology. Forty-seven of 69 recommendations (68.1%) were comprised of Strong/Low or Strong/Very Low variation requiring justification within five paradigmatic scenarios. Among distribution in the five paradigmatic scenarios of Strong/Low and Strong/Very Low recommendations, the most common paradigmatic scenario was life threatening situation (n = 20/47; 42.6%). Four Strong/Low or Strong/Very Low recommendations (4/47; 8.5%) were outside Grading of Recommendations Assessment, Development and Evaluation methodology. Conclusions: Among a large, diverse assembly of critical care guideline recommendations using Grading of Recommendations Assessment, Development and Evaluation methodology, the strength of evidence of a recommendation was generally associated with the quality of evidence. However, strong recommendations were not infrequently made in the absence of high/moderate quality of evidence. To improve clarity and uptake, future guideline statements may specify why such pairings were made, avoid such pairings when outside of Grading of Recommendations Assessment, Development and Evaluation criteria, and consider separate language for Expert Consensus recommendations (good practice statements).

Original languageEnglish (US)
Pages (from-to)109-113
Number of pages5
JournalCritical care medicine
Volume47
Issue number1
DOIs
StatePublished - Jan 1 2019

Fingerprint

Critical Care
Medicine
Guidelines
Consensus
Information Storage and Retrieval
Kidney Diseases
Communicable Diseases
Sepsis
Language

Keywords

  • critical care
  • epistemology
  • evidence-based medicine
  • GRADE
  • guidelines

ASJC Scopus subject areas

  • Critical Care and Intensive Care Medicine

Cite this

Above the GRADE : Evaluation of Guidelines in Critical Care Medicine. / Sims, Charles R.; Warner, Matthew; Stelfox, Henry Thomas; Hyder, Joseph A.

In: Critical care medicine, Vol. 47, No. 1, 01.01.2019, p. 109-113.

Research output: Contribution to journalReview article

Sims, Charles R. ; Warner, Matthew ; Stelfox, Henry Thomas ; Hyder, Joseph A. / Above the GRADE : Evaluation of Guidelines in Critical Care Medicine. In: Critical care medicine. 2019 ; Vol. 47, No. 1. pp. 109-113.
@article{dc9a8b94abbe4e42be363213c3e1af2e,
title = "Above the GRADE: Evaluation of Guidelines in Critical Care Medicine",
abstract = "Objectives: We examined recommendations within critical care guidelines to describe the pairing patterns for strength of recommendation and quality of evidence. We further identified recommendations where the reported strength of recommendation was strong while the reported quality of evidence was not high/moderate and then assessed whether such pairings were within five paradigmatic situations offered by Grading of Recommendations Assessment, Development and Evaluation methodology to justify such pairings. Data Sources and Extraction: We identified all clinical critical care guidelines published online from 2011 to 2017 by the Society of Critical Care Medicine along with individual guidelines published by Surviving Sepsis Campaign, Kidney Disease Improving Global Outcomes, American Society for Parenteral and Enteral Nutrition, and the Infectious Disease Society of America/American Thoracic Society. Data Synthesis: In all, 15 documents specifying 681 eligible recommendations demonstrated variation in strength of recommendation (strong n = 215 [31.6{\%}], weak n = 345 [50.7{\%}], none n = 121 [17.8{\%}]) and in quality of evidence (high n = 41 [6.0{\%}], moderate n = 151 [22.2{\%}], low/very low n = 298 [43.8{\%}], and Expert Consensus/none n = 191 [28.1{\%}]). Strength of recommendation and quality of evidence were positively correlated (ρ = 0.66; p < 0.0001). Of 215 strong recommendations, 69 (32.1{\%}) were discordantly paired with evidence other than high/moderate. Twenty-two of 69 (31.9{\%}) involved Strong/Expert Consensus recommendations, a category discouraged by Grading of Recommendations Assessment, Development and Evaluation methodology. Forty-seven of 69 recommendations (68.1{\%}) were comprised of Strong/Low or Strong/Very Low variation requiring justification within five paradigmatic scenarios. Among distribution in the five paradigmatic scenarios of Strong/Low and Strong/Very Low recommendations, the most common paradigmatic scenario was life threatening situation (n = 20/47; 42.6{\%}). Four Strong/Low or Strong/Very Low recommendations (4/47; 8.5{\%}) were outside Grading of Recommendations Assessment, Development and Evaluation methodology. Conclusions: Among a large, diverse assembly of critical care guideline recommendations using Grading of Recommendations Assessment, Development and Evaluation methodology, the strength of evidence of a recommendation was generally associated with the quality of evidence. However, strong recommendations were not infrequently made in the absence of high/moderate quality of evidence. To improve clarity and uptake, future guideline statements may specify why such pairings were made, avoid such pairings when outside of Grading of Recommendations Assessment, Development and Evaluation criteria, and consider separate language for Expert Consensus recommendations (good practice statements).",
keywords = "critical care, epistemology, evidence-based medicine, GRADE, guidelines",
author = "Sims, {Charles R.} and Matthew Warner and Stelfox, {Henry Thomas} and Hyder, {Joseph A.}",
year = "2019",
month = "1",
day = "1",
doi = "10.1097/CCM.0000000000003467",
language = "English (US)",
volume = "47",
pages = "109--113",
journal = "Critical Care Medicine",
issn = "0090-3493",
publisher = "Lippincott Williams and Wilkins",
number = "1",

}

TY - JOUR

T1 - Above the GRADE

T2 - Evaluation of Guidelines in Critical Care Medicine

AU - Sims, Charles R.

AU - Warner, Matthew

AU - Stelfox, Henry Thomas

AU - Hyder, Joseph A.

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Objectives: We examined recommendations within critical care guidelines to describe the pairing patterns for strength of recommendation and quality of evidence. We further identified recommendations where the reported strength of recommendation was strong while the reported quality of evidence was not high/moderate and then assessed whether such pairings were within five paradigmatic situations offered by Grading of Recommendations Assessment, Development and Evaluation methodology to justify such pairings. Data Sources and Extraction: We identified all clinical critical care guidelines published online from 2011 to 2017 by the Society of Critical Care Medicine along with individual guidelines published by Surviving Sepsis Campaign, Kidney Disease Improving Global Outcomes, American Society for Parenteral and Enteral Nutrition, and the Infectious Disease Society of America/American Thoracic Society. Data Synthesis: In all, 15 documents specifying 681 eligible recommendations demonstrated variation in strength of recommendation (strong n = 215 [31.6%], weak n = 345 [50.7%], none n = 121 [17.8%]) and in quality of evidence (high n = 41 [6.0%], moderate n = 151 [22.2%], low/very low n = 298 [43.8%], and Expert Consensus/none n = 191 [28.1%]). Strength of recommendation and quality of evidence were positively correlated (ρ = 0.66; p < 0.0001). Of 215 strong recommendations, 69 (32.1%) were discordantly paired with evidence other than high/moderate. Twenty-two of 69 (31.9%) involved Strong/Expert Consensus recommendations, a category discouraged by Grading of Recommendations Assessment, Development and Evaluation methodology. Forty-seven of 69 recommendations (68.1%) were comprised of Strong/Low or Strong/Very Low variation requiring justification within five paradigmatic scenarios. Among distribution in the five paradigmatic scenarios of Strong/Low and Strong/Very Low recommendations, the most common paradigmatic scenario was life threatening situation (n = 20/47; 42.6%). Four Strong/Low or Strong/Very Low recommendations (4/47; 8.5%) were outside Grading of Recommendations Assessment, Development and Evaluation methodology. Conclusions: Among a large, diverse assembly of critical care guideline recommendations using Grading of Recommendations Assessment, Development and Evaluation methodology, the strength of evidence of a recommendation was generally associated with the quality of evidence. However, strong recommendations were not infrequently made in the absence of high/moderate quality of evidence. To improve clarity and uptake, future guideline statements may specify why such pairings were made, avoid such pairings when outside of Grading of Recommendations Assessment, Development and Evaluation criteria, and consider separate language for Expert Consensus recommendations (good practice statements).

AB - Objectives: We examined recommendations within critical care guidelines to describe the pairing patterns for strength of recommendation and quality of evidence. We further identified recommendations where the reported strength of recommendation was strong while the reported quality of evidence was not high/moderate and then assessed whether such pairings were within five paradigmatic situations offered by Grading of Recommendations Assessment, Development and Evaluation methodology to justify such pairings. Data Sources and Extraction: We identified all clinical critical care guidelines published online from 2011 to 2017 by the Society of Critical Care Medicine along with individual guidelines published by Surviving Sepsis Campaign, Kidney Disease Improving Global Outcomes, American Society for Parenteral and Enteral Nutrition, and the Infectious Disease Society of America/American Thoracic Society. Data Synthesis: In all, 15 documents specifying 681 eligible recommendations demonstrated variation in strength of recommendation (strong n = 215 [31.6%], weak n = 345 [50.7%], none n = 121 [17.8%]) and in quality of evidence (high n = 41 [6.0%], moderate n = 151 [22.2%], low/very low n = 298 [43.8%], and Expert Consensus/none n = 191 [28.1%]). Strength of recommendation and quality of evidence were positively correlated (ρ = 0.66; p < 0.0001). Of 215 strong recommendations, 69 (32.1%) were discordantly paired with evidence other than high/moderate. Twenty-two of 69 (31.9%) involved Strong/Expert Consensus recommendations, a category discouraged by Grading of Recommendations Assessment, Development and Evaluation methodology. Forty-seven of 69 recommendations (68.1%) were comprised of Strong/Low or Strong/Very Low variation requiring justification within five paradigmatic scenarios. Among distribution in the five paradigmatic scenarios of Strong/Low and Strong/Very Low recommendations, the most common paradigmatic scenario was life threatening situation (n = 20/47; 42.6%). Four Strong/Low or Strong/Very Low recommendations (4/47; 8.5%) were outside Grading of Recommendations Assessment, Development and Evaluation methodology. Conclusions: Among a large, diverse assembly of critical care guideline recommendations using Grading of Recommendations Assessment, Development and Evaluation methodology, the strength of evidence of a recommendation was generally associated with the quality of evidence. However, strong recommendations were not infrequently made in the absence of high/moderate quality of evidence. To improve clarity and uptake, future guideline statements may specify why such pairings were made, avoid such pairings when outside of Grading of Recommendations Assessment, Development and Evaluation criteria, and consider separate language for Expert Consensus recommendations (good practice statements).

KW - critical care

KW - epistemology

KW - evidence-based medicine

KW - GRADE

KW - guidelines

UR - http://www.scopus.com/inward/record.url?scp=85058877330&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058877330&partnerID=8YFLogxK

U2 - 10.1097/CCM.0000000000003467

DO - 10.1097/CCM.0000000000003467

M3 - Review article

VL - 47

SP - 109

EP - 113

JO - Critical Care Medicine

JF - Critical Care Medicine

SN - 0090-3493

IS - 1

ER -