Feasibility of Using Real-World Data to Replicate Clinical Trial Evidence

Victoria L. Bartlett, Sanket S. Dhruva, Nilay D. Shah, Patrick Ryan, Joseph S. Ross

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Importance: Although randomized clinical trials are considered to be the criterion standard for generating clinical evidence, the use of real-world evidence to evaluate the efficacy and safety of medical interventions is gaining interest. Whether observational data can be used to address the same clinical questions being answered by traditional clinical trials is still unclear. Objective: To identify the number of clinical trials published in high-impact journals in 2017 that could be feasibly replicated using observational data from insurance claims and/or electronic health records (EHRs). Design, Setting, and Participants: In this cross-sectional analysis, PubMed was searched to identify all US-based clinical trials, regardless of randomization, published between January 1, 2017, and December 31, 2017, in the top 7 highest-impact general medical journals of 2017. Trials were excluded if they did not involve human participants, did not use end points that represented clinical outcomes among patients, were not characterized as clinical trials, and had no recruitment sites in the United States. Main Outcomes and Measures: The primary outcomes were the number and percentage of trials for which the intervention, indication, trial inclusion and exclusion criteria, and primary end points could be ascertained from insurance claims and/or EHR data. Results: Of the 220 US-based trials analyzed, 33 (15.0%) could be replicated using observational data because their intervention, indication, inclusion and exclusion criteria, and primary end points could be routinely ascertained from insurance claims and/or EHR data. Of the 220 trials, 86 (39.1%) had an intervention that could be ascertained from insurance claims and/or EHR data. Among the 86 trials, 62 (72.1%) had an indication that could be ascertained. Forty-five (72.6%) of 62 trials had at least 80% of inclusion and exclusion criteria data that could be ascertained. Of these 45 studies, 33 (73.3%) had at least 1 primary end point that could be ascertained. Conclusions and Relevance: This study found that only 15% of the US-based clinical trials published in high-impact journals in 2017 could be feasibly replicated through analysis of administrative claims or EHR data. This finding suggests the potential for real-world evidence to complement clinical trials, both by examining the concordance between randomized experiments and observational studies and by comparing the generalizability of the trial population with the real-world population of interest.

Original languageEnglish (US)
Pages (from-to)e1912869
JournalJAMA Network Open
Volume2
Issue number10
DOIs
StatePublished - Oct 2 2019

Fingerprint

Electronic Health Records
Clinical Trials
Insurance
Insurance Claim Review
Random Allocation
PubMed
Population
Observational Studies
Randomized Controlled Trials
Cross-Sectional Studies
Outcome Assessment (Health Care)
Safety

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Feasibility of Using Real-World Data to Replicate Clinical Trial Evidence. / Bartlett, Victoria L.; Dhruva, Sanket S.; Shah, Nilay D.; Ryan, Patrick; Ross, Joseph S.

In: JAMA Network Open, Vol. 2, No. 10, 02.10.2019, p. e1912869.

Research output: Contribution to journalArticle

Bartlett, Victoria L. ; Dhruva, Sanket S. ; Shah, Nilay D. ; Ryan, Patrick ; Ross, Joseph S. / Feasibility of Using Real-World Data to Replicate Clinical Trial Evidence. In: JAMA Network Open. 2019 ; Vol. 2, No. 10. pp. e1912869.
@article{b44649bd59ee4e839688272f816ecad8,
title = "Feasibility of Using Real-World Data to Replicate Clinical Trial Evidence",
abstract = "Importance: Although randomized clinical trials are considered to be the criterion standard for generating clinical evidence, the use of real-world evidence to evaluate the efficacy and safety of medical interventions is gaining interest. Whether observational data can be used to address the same clinical questions being answered by traditional clinical trials is still unclear. Objective: To identify the number of clinical trials published in high-impact journals in 2017 that could be feasibly replicated using observational data from insurance claims and/or electronic health records (EHRs). Design, Setting, and Participants: In this cross-sectional analysis, PubMed was searched to identify all US-based clinical trials, regardless of randomization, published between January 1, 2017, and December 31, 2017, in the top 7 highest-impact general medical journals of 2017. Trials were excluded if they did not involve human participants, did not use end points that represented clinical outcomes among patients, were not characterized as clinical trials, and had no recruitment sites in the United States. Main Outcomes and Measures: The primary outcomes were the number and percentage of trials for which the intervention, indication, trial inclusion and exclusion criteria, and primary end points could be ascertained from insurance claims and/or EHR data. Results: Of the 220 US-based trials analyzed, 33 (15.0{\%}) could be replicated using observational data because their intervention, indication, inclusion and exclusion criteria, and primary end points could be routinely ascertained from insurance claims and/or EHR data. Of the 220 trials, 86 (39.1{\%}) had an intervention that could be ascertained from insurance claims and/or EHR data. Among the 86 trials, 62 (72.1{\%}) had an indication that could be ascertained. Forty-five (72.6{\%}) of 62 trials had at least 80{\%} of inclusion and exclusion criteria data that could be ascertained. Of these 45 studies, 33 (73.3{\%}) had at least 1 primary end point that could be ascertained. Conclusions and Relevance: This study found that only 15{\%} of the US-based clinical trials published in high-impact journals in 2017 could be feasibly replicated through analysis of administrative claims or EHR data. This finding suggests the potential for real-world evidence to complement clinical trials, both by examining the concordance between randomized experiments and observational studies and by comparing the generalizability of the trial population with the real-world population of interest.",
author = "Bartlett, {Victoria L.} and Dhruva, {Sanket S.} and Shah, {Nilay D.} and Patrick Ryan and Ross, {Joseph S.}",
year = "2019",
month = "10",
day = "2",
doi = "10.1001/jamanetworkopen.2019.12869",
language = "English (US)",
volume = "2",
pages = "e1912869",
journal = "JAMA network open",
issn = "2574-3805",
publisher = "American Medical Association",
number = "10",

}

TY - JOUR

T1 - Feasibility of Using Real-World Data to Replicate Clinical Trial Evidence

AU - Bartlett, Victoria L.

AU - Dhruva, Sanket S.

AU - Shah, Nilay D.

AU - Ryan, Patrick

AU - Ross, Joseph S.

PY - 2019/10/2

Y1 - 2019/10/2

N2 - Importance: Although randomized clinical trials are considered to be the criterion standard for generating clinical evidence, the use of real-world evidence to evaluate the efficacy and safety of medical interventions is gaining interest. Whether observational data can be used to address the same clinical questions being answered by traditional clinical trials is still unclear. Objective: To identify the number of clinical trials published in high-impact journals in 2017 that could be feasibly replicated using observational data from insurance claims and/or electronic health records (EHRs). Design, Setting, and Participants: In this cross-sectional analysis, PubMed was searched to identify all US-based clinical trials, regardless of randomization, published between January 1, 2017, and December 31, 2017, in the top 7 highest-impact general medical journals of 2017. Trials were excluded if they did not involve human participants, did not use end points that represented clinical outcomes among patients, were not characterized as clinical trials, and had no recruitment sites in the United States. Main Outcomes and Measures: The primary outcomes were the number and percentage of trials for which the intervention, indication, trial inclusion and exclusion criteria, and primary end points could be ascertained from insurance claims and/or EHR data. Results: Of the 220 US-based trials analyzed, 33 (15.0%) could be replicated using observational data because their intervention, indication, inclusion and exclusion criteria, and primary end points could be routinely ascertained from insurance claims and/or EHR data. Of the 220 trials, 86 (39.1%) had an intervention that could be ascertained from insurance claims and/or EHR data. Among the 86 trials, 62 (72.1%) had an indication that could be ascertained. Forty-five (72.6%) of 62 trials had at least 80% of inclusion and exclusion criteria data that could be ascertained. Of these 45 studies, 33 (73.3%) had at least 1 primary end point that could be ascertained. Conclusions and Relevance: This study found that only 15% of the US-based clinical trials published in high-impact journals in 2017 could be feasibly replicated through analysis of administrative claims or EHR data. This finding suggests the potential for real-world evidence to complement clinical trials, both by examining the concordance between randomized experiments and observational studies and by comparing the generalizability of the trial population with the real-world population of interest.

AB - Importance: Although randomized clinical trials are considered to be the criterion standard for generating clinical evidence, the use of real-world evidence to evaluate the efficacy and safety of medical interventions is gaining interest. Whether observational data can be used to address the same clinical questions being answered by traditional clinical trials is still unclear. Objective: To identify the number of clinical trials published in high-impact journals in 2017 that could be feasibly replicated using observational data from insurance claims and/or electronic health records (EHRs). Design, Setting, and Participants: In this cross-sectional analysis, PubMed was searched to identify all US-based clinical trials, regardless of randomization, published between January 1, 2017, and December 31, 2017, in the top 7 highest-impact general medical journals of 2017. Trials were excluded if they did not involve human participants, did not use end points that represented clinical outcomes among patients, were not characterized as clinical trials, and had no recruitment sites in the United States. Main Outcomes and Measures: The primary outcomes were the number and percentage of trials for which the intervention, indication, trial inclusion and exclusion criteria, and primary end points could be ascertained from insurance claims and/or EHR data. Results: Of the 220 US-based trials analyzed, 33 (15.0%) could be replicated using observational data because their intervention, indication, inclusion and exclusion criteria, and primary end points could be routinely ascertained from insurance claims and/or EHR data. Of the 220 trials, 86 (39.1%) had an intervention that could be ascertained from insurance claims and/or EHR data. Among the 86 trials, 62 (72.1%) had an indication that could be ascertained. Forty-five (72.6%) of 62 trials had at least 80% of inclusion and exclusion criteria data that could be ascertained. Of these 45 studies, 33 (73.3%) had at least 1 primary end point that could be ascertained. Conclusions and Relevance: This study found that only 15% of the US-based clinical trials published in high-impact journals in 2017 could be feasibly replicated through analysis of administrative claims or EHR data. This finding suggests the potential for real-world evidence to complement clinical trials, both by examining the concordance between randomized experiments and observational studies and by comparing the generalizability of the trial population with the real-world population of interest.

UR - http://www.scopus.com/inward/record.url?scp=85073078097&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073078097&partnerID=8YFLogxK

U2 - 10.1001/jamanetworkopen.2019.12869

DO - 10.1001/jamanetworkopen.2019.12869

M3 - Article

C2 - 31596493

AN - SCOPUS:85073078097

VL - 2

SP - e1912869

JO - JAMA network open

JF - JAMA network open

SN - 2574-3805

IS - 10

ER -