Senior GRADE methodologists encounter challenges as part of WHO guideline development panels: An inductive content analysis

Paul E. Alexander, Shelly Anne Li, Michael R. Gionfriddo, Rebecca J. Stoltzfus, Ignacio Neumann, Juan Brito Campana, Benjamin Djulbegovic, Victor Manuel Montori, Holger J. Schünemann, Gordon H. Guyatt

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Background The World Health Organization (WHO) classifies a substantial proportion of their recommendations as strong despite low or very low confidence (certainty) in estimates of effect. Such discordant recommendations are often inconsistent with Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance. Objective To gain the perspective of senior WHO methodology chairs regarding panels' use of GRADE, particularly regarding discordant recommendations. Data sources Senior active GRADE methodologists who had served on at least two WHO panels and were an author on at least one peer-reviewed published article on GRADE methodology. Methods Five eligible methodologists participated in detailed semistructured interviews. Respondents answered questions regarding how they were viewed by other panelists and WHO leadership, and how they handled situations when panelists made discordant recommendations they felt were inappropriate. They also provided information on how the process can be improved. Interviews were recorded and transcribed, and inductive content analysis was used to derive codes, categories, and emergent themes. Results Three themes emerged from the interviews of five methodologists: (1) The perceived role of methodologists in the process, (2) Contributors to discordant recommendations, and (3) Strategies for improvement. Salient findings included (1) a perceived tension between methodologists and WHO panels as a result of panel members' resistance to adhering to GRADE guidance; (2) both financial and nonfinancial conflicts of interest among panel members as an explanation for discordant recommendations; and (3) the need for greater clarity of, and support for, the role of methodologists as co-chairs of panels. Conclusions These findings suggest that the role of the GRADE methodologist as a co-chair needs to be clarified by the WHO leadership. They further suggest the need for additional training for panelists, quality monitoring, and feedback to ensure optimal use of GRADE in guideline development at WHO.

Original languageEnglish (US)
Pages (from-to)123-128
Number of pages6
JournalJournal of Clinical Epidemiology
Volume70
DOIs
StatePublished - Feb 1 2016

Fingerprint

Guidelines
Interviews
Conflict of Interest
Information Storage and Retrieval

Keywords

  • Discordant recommendations
  • Financial conflicts of interest
  • GRADE
  • Guidelines
  • Low confidence
  • Methodologist
  • Nonfinancial conflicts of interest
  • Panel
  • Strong recommendation
  • WHO

ASJC Scopus subject areas

  • Epidemiology

Cite this

Senior GRADE methodologists encounter challenges as part of WHO guideline development panels : An inductive content analysis. / Alexander, Paul E.; Li, Shelly Anne; Gionfriddo, Michael R.; Stoltzfus, Rebecca J.; Neumann, Ignacio; Brito Campana, Juan; Djulbegovic, Benjamin; Montori, Victor Manuel; Schünemann, Holger J.; Guyatt, Gordon H.

In: Journal of Clinical Epidemiology, Vol. 70, 01.02.2016, p. 123-128.

Research output: Contribution to journalArticle

Alexander, Paul E. ; Li, Shelly Anne ; Gionfriddo, Michael R. ; Stoltzfus, Rebecca J. ; Neumann, Ignacio ; Brito Campana, Juan ; Djulbegovic, Benjamin ; Montori, Victor Manuel ; Schünemann, Holger J. ; Guyatt, Gordon H. / Senior GRADE methodologists encounter challenges as part of WHO guideline development panels : An inductive content analysis. In: Journal of Clinical Epidemiology. 2016 ; Vol. 70. pp. 123-128.
@article{87d1e510396d421698b2aa33388dcd51,
title = "Senior GRADE methodologists encounter challenges as part of WHO guideline development panels: An inductive content analysis",
abstract = "Background The World Health Organization (WHO) classifies a substantial proportion of their recommendations as strong despite low or very low confidence (certainty) in estimates of effect. Such discordant recommendations are often inconsistent with Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance. Objective To gain the perspective of senior WHO methodology chairs regarding panels' use of GRADE, particularly regarding discordant recommendations. Data sources Senior active GRADE methodologists who had served on at least two WHO panels and were an author on at least one peer-reviewed published article on GRADE methodology. Methods Five eligible methodologists participated in detailed semistructured interviews. Respondents answered questions regarding how they were viewed by other panelists and WHO leadership, and how they handled situations when panelists made discordant recommendations they felt were inappropriate. They also provided information on how the process can be improved. Interviews were recorded and transcribed, and inductive content analysis was used to derive codes, categories, and emergent themes. Results Three themes emerged from the interviews of five methodologists: (1) The perceived role of methodologists in the process, (2) Contributors to discordant recommendations, and (3) Strategies for improvement. Salient findings included (1) a perceived tension between methodologists and WHO panels as a result of panel members' resistance to adhering to GRADE guidance; (2) both financial and nonfinancial conflicts of interest among panel members as an explanation for discordant recommendations; and (3) the need for greater clarity of, and support for, the role of methodologists as co-chairs of panels. Conclusions These findings suggest that the role of the GRADE methodologist as a co-chair needs to be clarified by the WHO leadership. They further suggest the need for additional training for panelists, quality monitoring, and feedback to ensure optimal use of GRADE in guideline development at WHO.",
keywords = "Discordant recommendations, Financial conflicts of interest, GRADE, Guidelines, Low confidence, Methodologist, Nonfinancial conflicts of interest, Panel, Strong recommendation, WHO",
author = "Alexander, {Paul E.} and Li, {Shelly Anne} and Gionfriddo, {Michael R.} and Stoltzfus, {Rebecca J.} and Ignacio Neumann and {Brito Campana}, Juan and Benjamin Djulbegovic and Montori, {Victor Manuel} and Sch{\"u}nemann, {Holger J.} and Guyatt, {Gordon H.}",
year = "2016",
month = "2",
day = "1",
doi = "10.1016/j.jclinepi.2015.09.003",
language = "English (US)",
volume = "70",
pages = "123--128",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",

}

TY - JOUR

T1 - Senior GRADE methodologists encounter challenges as part of WHO guideline development panels

T2 - An inductive content analysis

AU - Alexander, Paul E.

AU - Li, Shelly Anne

AU - Gionfriddo, Michael R.

AU - Stoltzfus, Rebecca J.

AU - Neumann, Ignacio

AU - Brito Campana, Juan

AU - Djulbegovic, Benjamin

AU - Montori, Victor Manuel

AU - Schünemann, Holger J.

AU - Guyatt, Gordon H.

PY - 2016/2/1

Y1 - 2016/2/1

N2 - Background The World Health Organization (WHO) classifies a substantial proportion of their recommendations as strong despite low or very low confidence (certainty) in estimates of effect. Such discordant recommendations are often inconsistent with Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance. Objective To gain the perspective of senior WHO methodology chairs regarding panels' use of GRADE, particularly regarding discordant recommendations. Data sources Senior active GRADE methodologists who had served on at least two WHO panels and were an author on at least one peer-reviewed published article on GRADE methodology. Methods Five eligible methodologists participated in detailed semistructured interviews. Respondents answered questions regarding how they were viewed by other panelists and WHO leadership, and how they handled situations when panelists made discordant recommendations they felt were inappropriate. They also provided information on how the process can be improved. Interviews were recorded and transcribed, and inductive content analysis was used to derive codes, categories, and emergent themes. Results Three themes emerged from the interviews of five methodologists: (1) The perceived role of methodologists in the process, (2) Contributors to discordant recommendations, and (3) Strategies for improvement. Salient findings included (1) a perceived tension between methodologists and WHO panels as a result of panel members' resistance to adhering to GRADE guidance; (2) both financial and nonfinancial conflicts of interest among panel members as an explanation for discordant recommendations; and (3) the need for greater clarity of, and support for, the role of methodologists as co-chairs of panels. Conclusions These findings suggest that the role of the GRADE methodologist as a co-chair needs to be clarified by the WHO leadership. They further suggest the need for additional training for panelists, quality monitoring, and feedback to ensure optimal use of GRADE in guideline development at WHO.

AB - Background The World Health Organization (WHO) classifies a substantial proportion of their recommendations as strong despite low or very low confidence (certainty) in estimates of effect. Such discordant recommendations are often inconsistent with Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance. Objective To gain the perspective of senior WHO methodology chairs regarding panels' use of GRADE, particularly regarding discordant recommendations. Data sources Senior active GRADE methodologists who had served on at least two WHO panels and were an author on at least one peer-reviewed published article on GRADE methodology. Methods Five eligible methodologists participated in detailed semistructured interviews. Respondents answered questions regarding how they were viewed by other panelists and WHO leadership, and how they handled situations when panelists made discordant recommendations they felt were inappropriate. They also provided information on how the process can be improved. Interviews were recorded and transcribed, and inductive content analysis was used to derive codes, categories, and emergent themes. Results Three themes emerged from the interviews of five methodologists: (1) The perceived role of methodologists in the process, (2) Contributors to discordant recommendations, and (3) Strategies for improvement. Salient findings included (1) a perceived tension between methodologists and WHO panels as a result of panel members' resistance to adhering to GRADE guidance; (2) both financial and nonfinancial conflicts of interest among panel members as an explanation for discordant recommendations; and (3) the need for greater clarity of, and support for, the role of methodologists as co-chairs of panels. Conclusions These findings suggest that the role of the GRADE methodologist as a co-chair needs to be clarified by the WHO leadership. They further suggest the need for additional training for panelists, quality monitoring, and feedback to ensure optimal use of GRADE in guideline development at WHO.

KW - Discordant recommendations

KW - Financial conflicts of interest

KW - GRADE

KW - Guidelines

KW - Low confidence

KW - Methodologist

KW - Nonfinancial conflicts of interest

KW - Panel

KW - Strong recommendation

KW - WHO

UR - http://www.scopus.com/inward/record.url?scp=84957706017&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84957706017&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2015.09.003

DO - 10.1016/j.jclinepi.2015.09.003

M3 - Article

C2 - 26385188

AN - SCOPUS:84957706017

VL - 70

SP - 123

EP - 128

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -