New insights into the evaluation of randomized controlled trials for rare diseases over a long-term research horizon: a simulation study

Large sample sizes are required in randomized clinical trials designed to meet typical one-sided 2.5% α-level and 80% power. This may not be achievable when the disease is rare. We simulated a series of two-arm superiority trials over a 15-year period. The design parameters examined were the α-level and the number of trials conducted over the 15-year period (thus, trial sample size). Different disease severities and accrual rates were considered. The future treatment effect was characterized by its associated hazard rate; different hypotheses of how treatments improve over time were considered. We defined the total survival benefit as the relative difference of the hazard rates at year 15 versus year 0. The optimal design was defined by maximizing the expected total survival benefit, provided that the risk of selecting at year 15 a treatment inferior to the initial control treatment remains below 1%. Compared with two larger trials with typical one-sided 2.5% α-level, performing a series of small trials with relaxed α-levels leads on average to larger survival benefits over a 15-year research horizon, but also to higher risk of selecting a worse treatment at the end of the research period. Under reasonably optimistic assumptions regarding the future treatment effects, optimal designs outperform traditional ones when the disease is severe (baseline median survival ≤ 1 year) and the accrual is ≥100 patients per year, whereas no major improvement is observed in diseases with better prognosis. Trial designs aiming to maximize survival gain over a long research horizon across a series of trials are worth discussing in the context of rare diseases.