Systematic reviews with language restrictions and no author contact have lower overall credibility: A methodology study

Zhen Wang, Juan Brito Campana, Apostolos Tsapas, Marcio L. Griebeler, Fares Alahdab, Mohammad H Murad

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Background: High-quality systematic reviews (SRs) require rigorous approaches to identify, appraise, select, and synthesize research evidence relevant to a specific question. In this study, we evaluated the association between two steps in the conduct of an SR – restricting the search to English, and author contact for missing data – and the overall credibility of a SR. Methods: All SRs cited by the Endocrine Society's Clinical Practice Guidelines published from October 2006 through January 2012 were included. The main outcome was the overall A Measurement Tool to Assess Systematic Reviews (AMSTAR) score, as a surrogate of SR credibility. Nonparametric Kruskal–Wallis tests and multivariable linear regression models were used to investigate the association between language restriction, author contact for missing data, and the overall AMSTAR score. Results: In all, 69 SRs were included in the analysis. Only 31 SRs (45%) reported searching non-English literature, with an average AMSTAR score of 7.90 (standard deviation [SD] =1.64). SRs that reported language restriction received significantly lower AMSTAR scores (mean =5.25, SD =2.32) (P<0.001). Only 30 SRs (43%) reported contacting authors for missing data, and these received, on average, 2.59 more AMSTAR points (SD =1.95) than those who did not (P<0.001). In multivariable analyses, AMSTAR score was significantly correlated with language restriction (beta =-1.31, 95% confidence interval [CI]: -2.62, -0.01, P=0.05) and author contact for missing data (beta =2.16, 95% CI: 0.91, 3.41, P=0.001). However, after adjusting for compliance with reporting guidelines, language restriction was no longer significantly associated with the AMSTAR score. Conclusion: Fewer than half of the SRs conducted to support the clinical practice guidelines we examined reported contacting study authors or searched non–English literature. SRs that did not conduct these two steps had lower quality scores, suggesting the importance of these two steps for overall SR credibility.

Original languageEnglish (US)
Article numberA26
Pages (from-to)243-247
Number of pages5
JournalClinical Epidemiology
Volume7
DOIs
StatePublished - 2015

Fingerprint

Language
Practice Guidelines
Linear Models
Confidence Intervals
Literature
Guidelines
Research

Keywords

  • Evidence-based medicine
  • Quality of evidence
  • Research design
  • Validity

ASJC Scopus subject areas

  • Epidemiology

Cite this

Systematic reviews with language restrictions and no author contact have lower overall credibility : A methodology study. / Wang, Zhen; Brito Campana, Juan; Tsapas, Apostolos; Griebeler, Marcio L.; Alahdab, Fares; Murad, Mohammad H.

In: Clinical Epidemiology, Vol. 7, A26, 2015, p. 243-247.

Research output: Contribution to journalArticle

@article{d847fe2381bd4d29b7d6e3afb7720463,
title = "Systematic reviews with language restrictions and no author contact have lower overall credibility: A methodology study",
abstract = "Background: High-quality systematic reviews (SRs) require rigorous approaches to identify, appraise, select, and synthesize research evidence relevant to a specific question. In this study, we evaluated the association between two steps in the conduct of an SR – restricting the search to English, and author contact for missing data – and the overall credibility of a SR. Methods: All SRs cited by the Endocrine Society's Clinical Practice Guidelines published from October 2006 through January 2012 were included. The main outcome was the overall A Measurement Tool to Assess Systematic Reviews (AMSTAR) score, as a surrogate of SR credibility. Nonparametric Kruskal–Wallis tests and multivariable linear regression models were used to investigate the association between language restriction, author contact for missing data, and the overall AMSTAR score. Results: In all, 69 SRs were included in the analysis. Only 31 SRs (45{\%}) reported searching non-English literature, with an average AMSTAR score of 7.90 (standard deviation [SD] =1.64). SRs that reported language restriction received significantly lower AMSTAR scores (mean =5.25, SD =2.32) (P<0.001). Only 30 SRs (43{\%}) reported contacting authors for missing data, and these received, on average, 2.59 more AMSTAR points (SD =1.95) than those who did not (P<0.001). In multivariable analyses, AMSTAR score was significantly correlated with language restriction (beta =-1.31, 95{\%} confidence interval [CI]: -2.62, -0.01, P=0.05) and author contact for missing data (beta =2.16, 95{\%} CI: 0.91, 3.41, P=0.001). However, after adjusting for compliance with reporting guidelines, language restriction was no longer significantly associated with the AMSTAR score. Conclusion: Fewer than half of the SRs conducted to support the clinical practice guidelines we examined reported contacting study authors or searched non–English literature. SRs that did not conduct these two steps had lower quality scores, suggesting the importance of these two steps for overall SR credibility.",
keywords = "Evidence-based medicine, Quality of evidence, Research design, Validity",
author = "Zhen Wang and {Brito Campana}, Juan and Apostolos Tsapas and Griebeler, {Marcio L.} and Fares Alahdab and Murad, {Mohammad H}",
year = "2015",
doi = "10.2147/CLEP.S78879",
language = "English (US)",
volume = "7",
pages = "243--247",
journal = "Clinical Epidemiology",
issn = "1179-1349",
publisher = "Dove Medical Press Ltd.",

}

TY - JOUR

T1 - Systematic reviews with language restrictions and no author contact have lower overall credibility

T2 - A methodology study

AU - Wang, Zhen

AU - Brito Campana, Juan

AU - Tsapas, Apostolos

AU - Griebeler, Marcio L.

AU - Alahdab, Fares

AU - Murad, Mohammad H

PY - 2015

Y1 - 2015

N2 - Background: High-quality systematic reviews (SRs) require rigorous approaches to identify, appraise, select, and synthesize research evidence relevant to a specific question. In this study, we evaluated the association between two steps in the conduct of an SR – restricting the search to English, and author contact for missing data – and the overall credibility of a SR. Methods: All SRs cited by the Endocrine Society's Clinical Practice Guidelines published from October 2006 through January 2012 were included. The main outcome was the overall A Measurement Tool to Assess Systematic Reviews (AMSTAR) score, as a surrogate of SR credibility. Nonparametric Kruskal–Wallis tests and multivariable linear regression models were used to investigate the association between language restriction, author contact for missing data, and the overall AMSTAR score. Results: In all, 69 SRs were included in the analysis. Only 31 SRs (45%) reported searching non-English literature, with an average AMSTAR score of 7.90 (standard deviation [SD] =1.64). SRs that reported language restriction received significantly lower AMSTAR scores (mean =5.25, SD =2.32) (P<0.001). Only 30 SRs (43%) reported contacting authors for missing data, and these received, on average, 2.59 more AMSTAR points (SD =1.95) than those who did not (P<0.001). In multivariable analyses, AMSTAR score was significantly correlated with language restriction (beta =-1.31, 95% confidence interval [CI]: -2.62, -0.01, P=0.05) and author contact for missing data (beta =2.16, 95% CI: 0.91, 3.41, P=0.001). However, after adjusting for compliance with reporting guidelines, language restriction was no longer significantly associated with the AMSTAR score. Conclusion: Fewer than half of the SRs conducted to support the clinical practice guidelines we examined reported contacting study authors or searched non–English literature. SRs that did not conduct these two steps had lower quality scores, suggesting the importance of these two steps for overall SR credibility.

AB - Background: High-quality systematic reviews (SRs) require rigorous approaches to identify, appraise, select, and synthesize research evidence relevant to a specific question. In this study, we evaluated the association between two steps in the conduct of an SR – restricting the search to English, and author contact for missing data – and the overall credibility of a SR. Methods: All SRs cited by the Endocrine Society's Clinical Practice Guidelines published from October 2006 through January 2012 were included. The main outcome was the overall A Measurement Tool to Assess Systematic Reviews (AMSTAR) score, as a surrogate of SR credibility. Nonparametric Kruskal–Wallis tests and multivariable linear regression models were used to investigate the association between language restriction, author contact for missing data, and the overall AMSTAR score. Results: In all, 69 SRs were included in the analysis. Only 31 SRs (45%) reported searching non-English literature, with an average AMSTAR score of 7.90 (standard deviation [SD] =1.64). SRs that reported language restriction received significantly lower AMSTAR scores (mean =5.25, SD =2.32) (P<0.001). Only 30 SRs (43%) reported contacting authors for missing data, and these received, on average, 2.59 more AMSTAR points (SD =1.95) than those who did not (P<0.001). In multivariable analyses, AMSTAR score was significantly correlated with language restriction (beta =-1.31, 95% confidence interval [CI]: -2.62, -0.01, P=0.05) and author contact for missing data (beta =2.16, 95% CI: 0.91, 3.41, P=0.001). However, after adjusting for compliance with reporting guidelines, language restriction was no longer significantly associated with the AMSTAR score. Conclusion: Fewer than half of the SRs conducted to support the clinical practice guidelines we examined reported contacting study authors or searched non–English literature. SRs that did not conduct these two steps had lower quality scores, suggesting the importance of these two steps for overall SR credibility.

KW - Evidence-based medicine

KW - Quality of evidence

KW - Research design

KW - Validity

UR - http://www.scopus.com/inward/record.url?scp=84937002704&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84937002704&partnerID=8YFLogxK

U2 - 10.2147/CLEP.S78879

DO - 10.2147/CLEP.S78879

M3 - Article

AN - SCOPUS:84937002704

VL - 7

SP - 243

EP - 247

JO - Clinical Epidemiology

JF - Clinical Epidemiology

SN - 1179-1349

M1 - A26

ER -