Optimism bias leads to inconclusive results - An empirical study

Benjamin Djulbegovic, Ambuj Kumar, Anja Magazin, Anneke T. Schroen, Heloisa Soares, Iztok Hozo, Mike Clarke, Daniel Sargent, Michael J. Schell

Research output: Contribution to journalReview article

27 Scopus citations

Abstract

Objective: Optimism bias refers to unwarranted belief in the efficacy of new therapies. We assessed the impact of optimism bias on a proportion of trials that did not answer their research question successfully and explored whether poor accrual or optimism bias is responsible for inconclusive results. Study Design: Systematic review. Setting: Retrospective analysis of a consecutive-series phase III randomized controlled trials (RCTs) performed under the aegis of National Cancer Institute Cooperative groups. Results: Three hundred fifty-nine trials (374 comparisons) enrolling 150,232 patients were analyzed. Seventy percent (262 of 374) of the trials generated conclusive results according to the statistical criteria. Investigators made definitive statements related to the treatment preference in 73% (273 of 374) of studies. Investigators' judgments and statistical inferences were concordant in 75% (279 of 374) of trials. Investigators consistently overestimated their expected treatment effects but to a significantly larger extent for inconclusive trials. The median ratio of expected and observed hazard ratio or odds ratio was 1.34 (range: 0.19-15.40) in conclusive trials compared with 1.86 (range: 1.09-12.00) in inconclusive studies (P < 0.0001). Only 17% of the trials had treatment effects that matched original researchers' expectations. Conclusion: Formal statistical inference is sufficient to answer the research question in 75% of RCTs. The answers to the other 25% depend mostly on subjective judgments, which at times are in conflict with statistical inference. Optimism bias significantly contributes to inconclusive results.

Original languageEnglish (US)
Pages (from-to)583-593
Number of pages11
JournalJournal of Clinical Epidemiology
Volume64
Issue number6
DOIs
StatePublished - Jun 2011

Keywords

  • Bias
  • Inconclusive trials
  • Optimism bias
  • Randomized controlled trials
  • Study design
  • Systematic review

ASJC Scopus subject areas

  • Epidemiology

Fingerprint Dive into the research topics of 'Optimism bias leads to inconclusive results - An empirical study'. Together they form a unique fingerprint.

  • Cite this

    Djulbegovic, B., Kumar, A., Magazin, A., Schroen, A. T., Soares, H., Hozo, I., Clarke, M., Sargent, D., & Schell, M. J. (2011). Optimism bias leads to inconclusive results - An empirical study. Journal of Clinical Epidemiology, 64(6), 583-593. https://doi.org/10.1016/j.jclinepi.2010.09.007