TY - JOUR
T1 - Comparison of error rates in single-arm versus randomized phase II cancer clinical trials
AU - Tang, Hui
AU - Foster, Nathan R.
AU - Grothey, Axel
AU - Ansell, Stephen M.
AU - Goldberg, Richard M.
AU - Sargent, Daniel J.
PY - 2010/4/10
Y1 - 2010/4/10
N2 - Purpose: To improve the understanding of the appropriate design of phase II oncology clinical trials, we compared error rates in single-arm, historically controlled and randomized, concurrently controlled designs. Patients and Methods: We simulated error rates of both designs separately from individual patient data from a large colorectal cancer phase III trials and statistical models, which take into account random and systematic variation in historical control data. Results: In single-arm trials, false-positive error rates (type I error) were 2 to 4 times those projected when modest drift or patient selection effects (eg, 5% absolute shift in control response rate) were included in statistical models. The power of single-arm designs simulated using actual data was highly sensitive to the fraction of patients from treatment centers with high versus low patient volumes, the presence of patient selection effects or temporal drift in response rates, and random small-sample variation in historical controls. Increasing sample size did not correct the over optimism of single-arm studies. Randomized two-arm design conformed to planned error rates. Conclusion: Variability in historical control success rates, outcome drifts in patient populations over time, and/or patient selection effects can result in inaccurate false-positive and false-negative error rates in single-arm designs, but leave performance of the randomized two-arm design largely unaffected at the cost of 2 to 4 times the sample size compared with single-arm designs. Given a large enough patient pool, the randomized phase II designs provide a more accurate decision for screening agents before phase III testing.
AB - Purpose: To improve the understanding of the appropriate design of phase II oncology clinical trials, we compared error rates in single-arm, historically controlled and randomized, concurrently controlled designs. Patients and Methods: We simulated error rates of both designs separately from individual patient data from a large colorectal cancer phase III trials and statistical models, which take into account random and systematic variation in historical control data. Results: In single-arm trials, false-positive error rates (type I error) were 2 to 4 times those projected when modest drift or patient selection effects (eg, 5% absolute shift in control response rate) were included in statistical models. The power of single-arm designs simulated using actual data was highly sensitive to the fraction of patients from treatment centers with high versus low patient volumes, the presence of patient selection effects or temporal drift in response rates, and random small-sample variation in historical controls. Increasing sample size did not correct the over optimism of single-arm studies. Randomized two-arm design conformed to planned error rates. Conclusion: Variability in historical control success rates, outcome drifts in patient populations over time, and/or patient selection effects can result in inaccurate false-positive and false-negative error rates in single-arm designs, but leave performance of the randomized two-arm design largely unaffected at the cost of 2 to 4 times the sample size compared with single-arm designs. Given a large enough patient pool, the randomized phase II designs provide a more accurate decision for screening agents before phase III testing.
UR - http://www.scopus.com/inward/record.url?scp=77951639219&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=77951639219&partnerID=8YFLogxK
U2 - 10.1200/JCO.2009.25.5489
DO - 10.1200/JCO.2009.25.5489
M3 - Article
C2 - 20212253
AN - SCOPUS:77951639219
VL - 28
SP - 1936
EP - 1941
JO - Journal of Clinical Oncology
JF - Journal of Clinical Oncology
SN - 0732-183X
IS - 11
ER -