Two-stage adaptive cutoff design for building and validating a prognostic biomarker signature

Mei-Yin Polley, Eric Polley, Erich P. Huang, Boris Freidlin, Richard Simon

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Cancer biomarkers are frequently evaluated using archived specimens collected from previously conducted therapeutic trials. Routine collection and banking of high quality specimens is an expensive and time-consuming process. Therefore, care should be taken to preserve these precious resources. Here, we propose a novel two-stage adaptive cutoff design that affords the possibility to stop the biomarker study early if an evaluation of the model performance is unsatisfactory at an early stage, thereby allowing one to preserve the remaining specimens for future research. In addition, our design integrates important elements necessary to meet statistical rigor and practical demands for developing and validating a prognostic biomarker signature, including maintaining strict separation between the datasets used to build and evaluate the model and producing a locked-down signature to facilitate future validation. We conduct simulation studies to evaluate the operating characteristics of the proposed design. We show that under the null hypothesis when the model performance is deemed undesirable, the proposed design maintains type I error at the nominal level, has high probabilities of terminating the study early, and results in substantial savings in specimens. Under the alternative hypothesis, power is generally high when the total sample size and the targeted degree of improvement in prediction accuracy are reasonably large. We illustrate the use of the procedure with a dataset in patients with diffuse large-B-cell lymphoma. The practical aspects of the proposed designs are discussed. Published 2014. This article is a U.S. Government work and is in the public domain in the USA. Copyright

Original languageEnglish (US)
Pages (from-to)5097-5110
Number of pages14
JournalStatistics in Medicine
Volume33
Issue number29
DOIs
StatePublished - Jan 1 2014
Externally publishedYes

Fingerprint

Two-stage Design
Adaptive Design
Biomarkers
Signature
Lymphoma, Large B-Cell, Diffuse
Public Sector
Performance Model
Tumor Biomarkers
Sample Size
B Cells
Evaluate
Type I error
Banking
Operating Characteristics
Null hypothesis
Categorical or nominal
Cancer
Integrate
Simulation Study
Resources

Keywords

  • Biomarker validation
  • Cancer biomarker
  • Cross-validation
  • Early stopping
  • Two-stage design

ASJC Scopus subject areas

  • Epidemiology
  • Statistics and Probability

Cite this

Two-stage adaptive cutoff design for building and validating a prognostic biomarker signature. / Polley, Mei-Yin; Polley, Eric; Huang, Erich P.; Freidlin, Boris; Simon, Richard.

In: Statistics in Medicine, Vol. 33, No. 29, 01.01.2014, p. 5097-5110.

Research output: Contribution to journalArticle

Polley, Mei-Yin ; Polley, Eric ; Huang, Erich P. ; Freidlin, Boris ; Simon, Richard. / Two-stage adaptive cutoff design for building and validating a prognostic biomarker signature. In: Statistics in Medicine. 2014 ; Vol. 33, No. 29. pp. 5097-5110.
@article{0ed2ba06244e46b2961d8bf8dbd9668b,
title = "Two-stage adaptive cutoff design for building and validating a prognostic biomarker signature",
abstract = "Cancer biomarkers are frequently evaluated using archived specimens collected from previously conducted therapeutic trials. Routine collection and banking of high quality specimens is an expensive and time-consuming process. Therefore, care should be taken to preserve these precious resources. Here, we propose a novel two-stage adaptive cutoff design that affords the possibility to stop the biomarker study early if an evaluation of the model performance is unsatisfactory at an early stage, thereby allowing one to preserve the remaining specimens for future research. In addition, our design integrates important elements necessary to meet statistical rigor and practical demands for developing and validating a prognostic biomarker signature, including maintaining strict separation between the datasets used to build and evaluate the model and producing a locked-down signature to facilitate future validation. We conduct simulation studies to evaluate the operating characteristics of the proposed design. We show that under the null hypothesis when the model performance is deemed undesirable, the proposed design maintains type I error at the nominal level, has high probabilities of terminating the study early, and results in substantial savings in specimens. Under the alternative hypothesis, power is generally high when the total sample size and the targeted degree of improvement in prediction accuracy are reasonably large. We illustrate the use of the procedure with a dataset in patients with diffuse large-B-cell lymphoma. The practical aspects of the proposed designs are discussed. Published 2014. This article is a U.S. Government work and is in the public domain in the USA. Copyright",
keywords = "Biomarker validation, Cancer biomarker, Cross-validation, Early stopping, Two-stage design",
author = "Mei-Yin Polley and Eric Polley and Huang, {Erich P.} and Boris Freidlin and Richard Simon",
year = "2014",
month = "1",
day = "1",
doi = "10.1002/sim.6310",
language = "English (US)",
volume = "33",
pages = "5097--5110",
journal = "Statistics in Medicine",
issn = "0277-6715",
publisher = "John Wiley and Sons Ltd",
number = "29",

}

TY - JOUR

T1 - Two-stage adaptive cutoff design for building and validating a prognostic biomarker signature

AU - Polley, Mei-Yin

AU - Polley, Eric

AU - Huang, Erich P.

AU - Freidlin, Boris

AU - Simon, Richard

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Cancer biomarkers are frequently evaluated using archived specimens collected from previously conducted therapeutic trials. Routine collection and banking of high quality specimens is an expensive and time-consuming process. Therefore, care should be taken to preserve these precious resources. Here, we propose a novel two-stage adaptive cutoff design that affords the possibility to stop the biomarker study early if an evaluation of the model performance is unsatisfactory at an early stage, thereby allowing one to preserve the remaining specimens for future research. In addition, our design integrates important elements necessary to meet statistical rigor and practical demands for developing and validating a prognostic biomarker signature, including maintaining strict separation between the datasets used to build and evaluate the model and producing a locked-down signature to facilitate future validation. We conduct simulation studies to evaluate the operating characteristics of the proposed design. We show that under the null hypothesis when the model performance is deemed undesirable, the proposed design maintains type I error at the nominal level, has high probabilities of terminating the study early, and results in substantial savings in specimens. Under the alternative hypothesis, power is generally high when the total sample size and the targeted degree of improvement in prediction accuracy are reasonably large. We illustrate the use of the procedure with a dataset in patients with diffuse large-B-cell lymphoma. The practical aspects of the proposed designs are discussed. Published 2014. This article is a U.S. Government work and is in the public domain in the USA. Copyright

AB - Cancer biomarkers are frequently evaluated using archived specimens collected from previously conducted therapeutic trials. Routine collection and banking of high quality specimens is an expensive and time-consuming process. Therefore, care should be taken to preserve these precious resources. Here, we propose a novel two-stage adaptive cutoff design that affords the possibility to stop the biomarker study early if an evaluation of the model performance is unsatisfactory at an early stage, thereby allowing one to preserve the remaining specimens for future research. In addition, our design integrates important elements necessary to meet statistical rigor and practical demands for developing and validating a prognostic biomarker signature, including maintaining strict separation between the datasets used to build and evaluate the model and producing a locked-down signature to facilitate future validation. We conduct simulation studies to evaluate the operating characteristics of the proposed design. We show that under the null hypothesis when the model performance is deemed undesirable, the proposed design maintains type I error at the nominal level, has high probabilities of terminating the study early, and results in substantial savings in specimens. Under the alternative hypothesis, power is generally high when the total sample size and the targeted degree of improvement in prediction accuracy are reasonably large. We illustrate the use of the procedure with a dataset in patients with diffuse large-B-cell lymphoma. The practical aspects of the proposed designs are discussed. Published 2014. This article is a U.S. Government work and is in the public domain in the USA. Copyright

KW - Biomarker validation

KW - Cancer biomarker

KW - Cross-validation

KW - Early stopping

KW - Two-stage design

UR - http://www.scopus.com/inward/record.url?scp=84920175027&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84920175027&partnerID=8YFLogxK

U2 - 10.1002/sim.6310

DO - 10.1002/sim.6310

M3 - Article

C2 - 25263614

AN - SCOPUS:84920175027

VL - 33

SP - 5097

EP - 5110

JO - Statistics in Medicine

JF - Statistics in Medicine

SN - 0277-6715

IS - 29

ER -