Association between funding and quality of published medical education research

Darcy A. Reed, David Allan Cook, Thomas J. Beckman, Rachel B. Levine, David E. Kern, Scott M. Wright

Research output: Contribution to journalArticle

313 Citations (Scopus)

Abstract

Context: Methodological shortcomings in medical education research are often attributed to insufficient funding, yet an association between funding and study quality has not been established. Objectives: To develop and evaluate an instrument for measuring the quality of education research studies and to assess the relationship between funding and study quality. Design, Setting, and Participants: Internal consistency, interrater and intrarater reliability, and criterion validity were determined for a 10-item medical education research study quality instrument (MERSQI). This was applied to 210 medical education research studies published in 13 peer-reviewed journals between September 1, 2002, and December 31, 2003. The amount of funding obtained per study and the publication record of the first author were determined by survey. Main Outcome Measures: Study quality as measured by the MERSQI (potential maximum total score, 18; maximum domain score, 3), amount of funding per study, and previous publications by the first author. Results: The mean MERSQI score was 9.95 (SD, 2.34; range, 5-16). Mean domain scores were highest for data analysis (2.58) and lowest for validity (0.69). Intraclass correlation coefficient ranges for interrater and intrarater reliability were 0.72 to 0.98 and 0.78 to 0.998, respectively. Total MERSQI scores were associated with expert quality ratings (Spearman ρ, 0.73; 95% confidence interval [CI], 0.56-0.84; P<.001), 3-year citation rate (0.8 increase in score per 10 citations; 95% CI, 0.03-1.30; P=.003), and journal impact factor (1.0 increase in score per 6-unit increase in impact factor; 95% CI, 0.34-1.56; P=.003). In multivariate analysis, MERSQI scores were independently associated with study funding of $20 000 or more (0.95 increase in score; 95% CI, 0.22-1.86; P=.045) and previous medical education publications by the first author (1.07 increase in score per 20 publications; 95% CI, 0.15-2.23; P=.047). Conclusion: The quality of published medical education research is associated with study funding.

Original languageEnglish (US)
Pages (from-to)1002-1009
Number of pages8
JournalJournal of the American Medical Association
Volume298
Issue number9
DOIs
StatePublished - Sep 5 2007

Fingerprint

Medical Education
Biomedical Research
Confidence Intervals
Publications
Journal Impact Factor
Outcome Assessment (Health Care)
Reproducibility of Results
Multivariate Analysis
Education
Research

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Association between funding and quality of published medical education research. / Reed, Darcy A.; Cook, David Allan; Beckman, Thomas J.; Levine, Rachel B.; Kern, David E.; Wright, Scott M.

In: Journal of the American Medical Association, Vol. 298, No. 9, 05.09.2007, p. 1002-1009.

Research output: Contribution to journalArticle

Reed, Darcy A. ; Cook, David Allan ; Beckman, Thomas J. ; Levine, Rachel B. ; Kern, David E. ; Wright, Scott M. / Association between funding and quality of published medical education research. In: Journal of the American Medical Association. 2007 ; Vol. 298, No. 9. pp. 1002-1009.
@article{927aef5524cb48ffbf3524c5e8e073c4,
title = "Association between funding and quality of published medical education research",
abstract = "Context: Methodological shortcomings in medical education research are often attributed to insufficient funding, yet an association between funding and study quality has not been established. Objectives: To develop and evaluate an instrument for measuring the quality of education research studies and to assess the relationship between funding and study quality. Design, Setting, and Participants: Internal consistency, interrater and intrarater reliability, and criterion validity were determined for a 10-item medical education research study quality instrument (MERSQI). This was applied to 210 medical education research studies published in 13 peer-reviewed journals between September 1, 2002, and December 31, 2003. The amount of funding obtained per study and the publication record of the first author were determined by survey. Main Outcome Measures: Study quality as measured by the MERSQI (potential maximum total score, 18; maximum domain score, 3), amount of funding per study, and previous publications by the first author. Results: The mean MERSQI score was 9.95 (SD, 2.34; range, 5-16). Mean domain scores were highest for data analysis (2.58) and lowest for validity (0.69). Intraclass correlation coefficient ranges for interrater and intrarater reliability were 0.72 to 0.98 and 0.78 to 0.998, respectively. Total MERSQI scores were associated with expert quality ratings (Spearman ρ, 0.73; 95{\%} confidence interval [CI], 0.56-0.84; P<.001), 3-year citation rate (0.8 increase in score per 10 citations; 95{\%} CI, 0.03-1.30; P=.003), and journal impact factor (1.0 increase in score per 6-unit increase in impact factor; 95{\%} CI, 0.34-1.56; P=.003). In multivariate analysis, MERSQI scores were independently associated with study funding of $20 000 or more (0.95 increase in score; 95{\%} CI, 0.22-1.86; P=.045) and previous medical education publications by the first author (1.07 increase in score per 20 publications; 95{\%} CI, 0.15-2.23; P=.047). Conclusion: The quality of published medical education research is associated with study funding.",
author = "Reed, {Darcy A.} and Cook, {David Allan} and Beckman, {Thomas J.} and Levine, {Rachel B.} and Kern, {David E.} and Wright, {Scott M.}",
year = "2007",
month = "9",
day = "5",
doi = "10.1001/jama.298.9.1002",
language = "English (US)",
volume = "298",
pages = "1002--1009",
journal = "JAMA - Journal of the American Medical Association",
issn = "0002-9955",
publisher = "American Medical Association",
number = "9",

}

TY - JOUR

T1 - Association between funding and quality of published medical education research

AU - Reed, Darcy A.

AU - Cook, David Allan

AU - Beckman, Thomas J.

AU - Levine, Rachel B.

AU - Kern, David E.

AU - Wright, Scott M.

PY - 2007/9/5

Y1 - 2007/9/5

N2 - Context: Methodological shortcomings in medical education research are often attributed to insufficient funding, yet an association between funding and study quality has not been established. Objectives: To develop and evaluate an instrument for measuring the quality of education research studies and to assess the relationship between funding and study quality. Design, Setting, and Participants: Internal consistency, interrater and intrarater reliability, and criterion validity were determined for a 10-item medical education research study quality instrument (MERSQI). This was applied to 210 medical education research studies published in 13 peer-reviewed journals between September 1, 2002, and December 31, 2003. The amount of funding obtained per study and the publication record of the first author were determined by survey. Main Outcome Measures: Study quality as measured by the MERSQI (potential maximum total score, 18; maximum domain score, 3), amount of funding per study, and previous publications by the first author. Results: The mean MERSQI score was 9.95 (SD, 2.34; range, 5-16). Mean domain scores were highest for data analysis (2.58) and lowest for validity (0.69). Intraclass correlation coefficient ranges for interrater and intrarater reliability were 0.72 to 0.98 and 0.78 to 0.998, respectively. Total MERSQI scores were associated with expert quality ratings (Spearman ρ, 0.73; 95% confidence interval [CI], 0.56-0.84; P<.001), 3-year citation rate (0.8 increase in score per 10 citations; 95% CI, 0.03-1.30; P=.003), and journal impact factor (1.0 increase in score per 6-unit increase in impact factor; 95% CI, 0.34-1.56; P=.003). In multivariate analysis, MERSQI scores were independently associated with study funding of $20 000 or more (0.95 increase in score; 95% CI, 0.22-1.86; P=.045) and previous medical education publications by the first author (1.07 increase in score per 20 publications; 95% CI, 0.15-2.23; P=.047). Conclusion: The quality of published medical education research is associated with study funding.

AB - Context: Methodological shortcomings in medical education research are often attributed to insufficient funding, yet an association between funding and study quality has not been established. Objectives: To develop and evaluate an instrument for measuring the quality of education research studies and to assess the relationship between funding and study quality. Design, Setting, and Participants: Internal consistency, interrater and intrarater reliability, and criterion validity were determined for a 10-item medical education research study quality instrument (MERSQI). This was applied to 210 medical education research studies published in 13 peer-reviewed journals between September 1, 2002, and December 31, 2003. The amount of funding obtained per study and the publication record of the first author were determined by survey. Main Outcome Measures: Study quality as measured by the MERSQI (potential maximum total score, 18; maximum domain score, 3), amount of funding per study, and previous publications by the first author. Results: The mean MERSQI score was 9.95 (SD, 2.34; range, 5-16). Mean domain scores were highest for data analysis (2.58) and lowest for validity (0.69). Intraclass correlation coefficient ranges for interrater and intrarater reliability were 0.72 to 0.98 and 0.78 to 0.998, respectively. Total MERSQI scores were associated with expert quality ratings (Spearman ρ, 0.73; 95% confidence interval [CI], 0.56-0.84; P<.001), 3-year citation rate (0.8 increase in score per 10 citations; 95% CI, 0.03-1.30; P=.003), and journal impact factor (1.0 increase in score per 6-unit increase in impact factor; 95% CI, 0.34-1.56; P=.003). In multivariate analysis, MERSQI scores were independently associated with study funding of $20 000 or more (0.95 increase in score; 95% CI, 0.22-1.86; P=.045) and previous medical education publications by the first author (1.07 increase in score per 20 publications; 95% CI, 0.15-2.23; P=.047). Conclusion: The quality of published medical education research is associated with study funding.

UR - http://www.scopus.com/inward/record.url?scp=34548430952&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34548430952&partnerID=8YFLogxK

U2 - 10.1001/jama.298.9.1002

DO - 10.1001/jama.298.9.1002

M3 - Article

C2 - 17785645

AN - SCOPUS:34548430952

VL - 298

SP - 1002

EP - 1009

JO - JAMA - Journal of the American Medical Association

JF - JAMA - Journal of the American Medical Association

SN - 0002-9955

IS - 9

ER -