Adaptive Clinical Trial Design: From Simple Dose-Finding Trials to Large-Scale Personalized Medicine Trials

While there’s great excitement about the potential of personalized medicine to improve care – particularly in oncology – there’s also a healthy dose of pessimism regarding the cost of clinical trials needed to bring optimal, targeted therapies to market. While many researchers believe that personalized medicine is the future of drug development and patient care, there is uncertainty about cost: Will attempts to develop personalized medicines push drug development and healthcare costs higher, or, as some researchers believe, will they lower overall healthcare costs once new targeted therapies are proved safe and effective? Although there have been notable successes, the fulfillment of such promises has been inconsistent. So how can investigators best move forward in developing targeted therapies that may spur even higher development costs? One reasonable strategy is to employ adaptive trial designs, which allow for a prospectively planned modification of certain aspect(s) of the trial design at interim stages, aim to improve the efficiency of a trial and increase the chance of success, while enhancing investigators’ understanding of the effect of the treatment.

Types of design adaptations include modifications of study eligibility criteria, randomization procedure, sample size, primary and secondary end points, treatment allocation, number of interim analyses, dose levels and duration, as well as methods of statistical analysis.

Many critical parameters used in planning a trial are estimated based on certain assumptions, such as response rates, standard deviations, and population means, because of an incomplete or inadequate understanding of these elements. Consequently, a trial may fail to achieve its goal when these prespecified estimates or assumptions substantially deviate from the truth. In many cases, investigators have a good understanding of an investigational drug only after the data have been collected and unblinded. It is not uncommon for investigators to discover only then that the dosage was suboptimal or that some arms of the trial proved unnecessary and could have been dropped early in the trial. However, within the structure of a nonadaptive clinical trial design, investigators can do little to address such limitations without harming the validity of the trial.

Over the past 20 years, such concerns have stimulated a tremendous effort to improve the efficiency of trial designs. A common theme has been a move toward adaptive designs. Use of the term “adaptive” has a long history in clinical trial literature. Cornfield and colleagues1 proposed an analytic approach for adaptive trials motivated by the 2-armed bandit model,² which aims to maximize the number of patients assigned to the more promising of 2 treatments. Zelen3 first introduced the concept of the play-the-winner rule for the same purpose. Wei and Durham⁴ proposed their play-the-winner rule for the randomization procedure in a sequential trial setting as an improvement of Zelen’s design. Today, the umbrella of adaptive design covers many approaches to introducing some degree of flexibility into a trial, ranging from the basic 3+3 phase 1 trial design for dose finding to cutting-edge biomarker adaptive design. The goal is to incorporate learning from running a trial into its later stages, so that the trial design can be corrected or improved when evidence suggests misspecifications have occurred in the trial’s planning phase. Appropriately, the FDA draft guidance on adaptive clinical trials defines an adaptive design clinical study as “a study that includes a prospectively planned opportunity for modification of one or more specified aspects of the study design and hypotheses based on analysis of data (usually interim data) from subjects in the study.”

Adaptive design offers the potential to yield more information about the investigational treatment than would otherwise be feasible within the time and resources allowed for a particular trial. An adaptive design may allow investigators to discontinue the data collection of 1 or more arms when the current cumulated data show ineffective for the arm(s), thereby reducing cost and time spent on treatments that are not promising based on the learn-and-confirm model, without decreasing the useful information gained of the overall trial. In addition, adaptive design trials may improve the understanding of the dose-response relationship using approaches such as continual reassessment method.^5,6

Type of Adaptive Designs

Some researchers categorize adaptive trial designs based on the rules for adaptations. There are roughly 4 categories: allocation rules, sampling rule, stopping rule, and decision rule. Allocation rules define how patients will be allocated to different arms in a trial. The sampling rule defines how many patients will be enrolled at the next stage. The stopping rule considers when to stop the trial for reasons such as efficacy, futility, harm, or safety. The decision rule refers to design modifications that are not covered by the previous 3 rules, including change of end point, trial hypothesis, and statistical analysis plan. The most widely used adaptive design methods include adaptive dose-finding design, adaptive randomization design, sample size reestimation design, group sequential design, adaptive seamless phase 2/3 design, adaptive treatment selection design, and biomarker design. Each of these design methods is described and discussed below.

Adaptive Dose-Finding Design

The simplest form of adaptive dose-finding trials is the 3+3 phase 1 trial design, which is commonly used in phase 1 oncology trials for finding a maximum tolerable dose (MTD). In a 3+3 trial, 3 patients enter the trial and start at a relatively low dose. If no dose-limiting toxicity (DLT) is observed, another 3 patients are added to the trial at a higher dose. If 1 patient in the first cohort experiences DLT, the 3 additional patients are added at the initial dose. If only 1 of the 6 patients experiences DLT, the dose escalation continues to the next higher level. If 2 or more patients experience DLT, the next lower dose is claimed to be the MTD. There are many variations on this type of design, as the 3+3 design can be generalized to m+n designs7 with different numbers of patients treated at each stage, or designs with different numbers of stages and end points. For example, a recently published study on treatment of metastatic melanoma conducted a phase 1 dose-escalation trial involving an m+n design transitioning seamlessly to a phase 2 design with the maximum dose that could be used without causing adverse effects.⁸

Generally, these approaches are often found to be inefficient and tend to underestimate the MTD, especially when the initial dose is too low, which necessitates large numbers of escalation steps, including several noninformative doses.⁷ Due to potential high toxicity of an investigational drug, many phase 1 trials have a very small number of patients at each dose level, particularly in the early stages. Such small sample sizes are clearly one of the sources of uncertainty (ie, high false-negative rate) regarding how precisely the toxicity rate at each stage and the MTD are estimated.

The continual reassessment method (CRM) was developed to address some of these problems and has drawn much attention. Many variations of CRM have been published and discussed, including both frequentist and Bayesian approaches. Assuming that the DLTs are binary outcome and that there is a monotonic relationship between dose and toxicity, the general idea behind the method is to assume a prior dose-response curve and the estimated dose-response relationship is updated after each patient’s outcome is observed, so that each patient’s dose is based on the cumulative information about how previously enrolled patients tolerated the drug. In a Bayesian framework, the prior curve represents the investigator’s prior knowledge about the dose response, and the accumulating information from previously enrolled patients can be constructed into the likelihood function. The resulting posterior probability will be the estimate of dose-response curve. This allows the investigators to optimize the trial by means of the adaptation to the cumulative data of an ongoing trial in a Bayesian framework. Compared with the conventional 3+3 design, the Bayesian CRM estimates the MTD more accurately, as it assigns more patients near MTD. Two possible downsides to the Bayesian CRM are: 1) it can increase the computational complexity, and 2) it can escalate doses too quickly. Several modified approaches have been developed in an attempt to overcome these problems. In addition to CRM designs, Ji’s design and modified Ji’s design, both Bayesian approaches, are also commonly used. More complicated adaptive dose-finding designs include designs for general monotonic response, and designs for U-shaped dose response are also available.

Example: Adaptive Trials for Dose-Finding Considering Both Efficacy and Safety

Two years ago, Berry and colleagues⁹ introduced their 2-stage adaptive dose-finding trial design with a Bayesian framework. The purpose of this design is to evaluate both efficacy and safety of an investigational drug at possible multiple doses. Initially, patients were randomized equally to 1 of the 4 arms: placebo, active control, treatment at a low dose, and treatment at a medium dose. The maximum number of patients allowed to enter stage I is 240. Based on evidence of efficacy and safety estimated using dose-response models, a decision will be made at each interim analysis to either terminate the trial or move to the second stage. If the trial moves to the second stage, a number of additional doses will be assessed. Patients will be allocated to placebo, treatment at up to 4 different dose levels (very low, low, medium, high), or proportionally to active control in an attempt to achieve equal allocation to all remaining arms at the end of the trial. Enrollment in the high-dose treatment arm depends on the observed toxicities during stage I, and enrollment in the very low-dose treatment arm depends on the observed efficacy from the low-dose treatment arm. The enrollment of stage II patients will continue to the number of the total sample size of the trial (N=500) if not all treatment arms were dropped. In the actual trial, the trial was terminated by the end of stage I after entering 199 patients based on the predictive probabilities that the low/medium treatment arm performs better than the control. One limitation of this trial design mentioned by the authors is that the highest dose is not evaluated at the beginning of the trial because of a relatively narrow prespecified dose range, limiting the potential efficiency gains with a fully responsive-adaptive design.

Adaptive Randomization Design

Adaptive randomization allows a design to modify randomization schedules by adjusting the probability of assigning patients to treatment groups based on the accumulated data of previously enrolled patients. Common types of adaptive randomization designs include treatment-adaptive randomization, covariate-adaptive randomization, and outcome-adaptive randomization. Treatment-adaptive randomization aims to achieve a more balanced treatment assignment by tuning the current allocation ratio between treatment arms. The most commonly used approaches include block randomization and the biased-coin method. Covariate-adaptive randomization adjusts the probability of assigning patients to any particular group based on the balance situation of 1 or more important covariates between treatment groups. One widely applied method is minimization randomization, which improves the trial’s statistical power but may cause bias since the investigator may be able to decode the randomization schedule. Outcome-adaptive randomization designs assign patients to treatment groups based on response of previous patients. One well-known method is the play-the-winner rule, which involves a dichotomous process. If the outcome of the previous patient is a success, then the same treatment will be assigned to the next patient; otherwise the patient will be assigned the other treatment. This strategy may not be feasible for trials in a disease with a relatively long response time because the next patient assignment depends on the outcome of the previous patients. A randomized play-the-winner rule was proposed to overcome the drawback of the long deterministic process by allocating patients. Although outcome-adaptive randomization could increase the likelihood of success, it has the limitation that the investigator may be able to guess the next treatment assignment based on current response rates, thereby biasing conclusions. In addition, there is the possibility that the patient allocation patterns could change over the course of the trial, when the trial is not effectively blinded, because of the knowledge that the randomization is favoring the treatment arm. Block stratification is used to minimize the bias and unbalanced allocation, especially if the trial is not placebo controlled. In a recent of the Journal of Clinical Oncology, there is an interesting ongoing discussion about whether the advantages of outcome-adaptive randomization overweigh its disadvantages or vice versa.^10,11

Sample Size Reestimation Design

In a nonadaptive setting, the sample size is planned prior to the implementation of a trial and is a fixed element. However, it is well recognized that estimation of sample size in clinical trials involve knowledge of treatment effect and variability, which are usually unknown at the planning stage of the trial. The basic idea behind sample size reestimation design is to preserve the statistical power in case of misspecification of the treatment effect or variance to avoid an underpowered trial or oversized trial. The sample size reestimation design allows for recalculating the required sample size based on the accumulating data at interim analyses, either with or without unblinding. Blinded sample size reestimation uses interim data without unblinding treatment assignment to provide an updated estimate of a nuisance parameter, usually the variance for continuous outcomes or the underlying event rate for binary outcomes, so that the required sample size can be adjusted based on the estimate. The number of such adjustments (usually once) and at what time points the interim analysis will be performed should be prespecified. Unblinded sample size reestimation adjusts the sample size at interim analyses based on unblinded interim results or other factors such as external information. It should be noted that the observed difference at interim analyses is often based on a small number of patients. Since sample size calculation is very sensitive to the effect size used, and treatment effect sizes at interim analyses can be highly variable, sample size estimation at interim stage may be unreliable and can be misleading.

Group Sequential Design

Group sequential design offers opportunities for early termination of a trial for either safety or efficacy reasons, allowing the sample size to be reduced. At each interim stage, all the accumulated data up to that point are analyzed, and a decision is made whether the trial is stopped or continued. Such decisions are made at interim analyses, and various stopping rules are available in the literature. Most methods attempt to control the overall type I error rate and to terminate the trial when there is neither enough beneficial treatment effect nor sufficient efficacy observed, using approaches such as alpha-spending rules.^12-15 If the implementation of a group sequential trial involves unblinding the interim results and analyzing the interim treatment effect, it can raise concerns of potential bias. The FDA recommends that the analyses be carried out either externally to the trial’s sponsor or by a group within the sponsor that is unequivocally separated from all other parties to the trial. In addition, some group sequential design approaches may not be able to control the overall type I error rate if the target patient population has been shifted due to additional adaptations or protocol amendments.

Adaptive Treatment Selection Designs

Play-the-Winner

As previously mentioned, the play-the-winner rule uses a simple probability model to randomize patients sequentially to the treatment arm(s). The first patient is randomly assigned to either treatment, with subsequent treatment assignments based on the response of the first patient; the next patient will be assigned to the treatment arm having the highest empirical rate. Play-the-winner approaches are used with the expectation that they would be statistically and ethically superior to simple randomization in the sense that more patients are treated with the better treatment at the end of the trial (based on the accumulated data in a particular trial). However, without sufficient safety data and a carefully developed statistical analysis plan, it is possible to assign more patients to a treatment that is more efficacious but also more toxic.

Drop-the-Loser

Similarly, a drop-the-loser design involves carrying out an interim analysis and drop 1 or more arms that are not promising, so that more patients can be assigned to better treatments.^16,17 It also allows dropping ineffective or high-toxicity treatment arm(s) and therefore may be in the interest of patient safety. The drop-the-loser design is often used to narrow the treatment arms or multiple doses with 2 stages. At the end of the first stage, the inferior arms/doses will be dropped, and the remaining arms/doses will proceed to the second stage. In practice, a 2-stage drop-the-loser trial is usually powered for the second stage, so there may not be sufficient statistical power for the interim analysis performed at the end of the first stage. Precision analysis (confidence interval calculations) is often used to make the decision of dropping the losers.

Treatment Switching

Another type of adaptive treatment selection design allows the patient to switch from one treatment arm to another based on prognosis and/or investigator’s judgment, often for safety or efficacy reasons. Most commonly, patients switch from the control arm to the intervention arm if there is lack of responses. In cancer trials, a switch may also occur when a patient’s disease progresses to a more severe grade, spurring a shift to an alternative treatment as a last resort. Statistical analysis of such trials may be a challenge because of the complexity caused by treatment switching. For example, patient survival rate will be very difficult to estimate when a large portion of patients switch to the other treatment. Treatment switching may also lead to a change in study hypotheses.

Example: Adaptive Treatment Allocation

A trial was conducted at the MD Anderson Cancer Center in the 1990s to study troxacitabine-based regimens as induction therapy in the treatment of acute myeloid leukemias.18 Initially, 75 patients were randomly assigned to 1 of 3 treatment arms at the following dosages: idarubicin 12 mg/m2 IV daily for 3 days and ara-C 1.5 g/m2 IV over 2 hours daily for 3 days versus troxacitabine 6 mg/m2 IV daily for 5 days and ara-C 1 g/m2 IV over 2 hours daily for 5 days versus troxacitabine 4 mg/m2  IV daily for 5 days and idarubicin 9 mg/m2  IV daily for 3 days with equal assigning probabilities (ie, P=1/3). The assigning probability of patients to treatment is adjusted based on the accumulated outcome responses (ie, complete response without nonhematologic grade 4 toxicity by 50 days) of patients previously enrolled in favor of arms that demonstrated better perfor­mance. This outcome-dependent randomization trial was designed in the spirit of the drop-the-loser rule. More specifically, a treatment arm will be dropped from the randomization if, at any time, the probability that the arm had a shorter time to response than the control or that the other treatment arm is greater than some prespecified threshold. In the actual trial, arm 3 (troxa­citabine/idarubicin) was dropped after treating 24 patients and observing the response of 21 other patients, because the control outperformed both arms (response rate: 56% for control, 43% for arm 1, and 0% for arm 2). The assignment probability for the 25th patient was 0.87 to the control and 0.13 to arm 1. Arm 1 was also dropped later after 34 patients had been treated, so the trial was stopped (response rate: 56% for control, 27% for arm 1, and 0% for arm 2). This trial used a Bayesian adaptive randomization framework in which the assigning probability is adjusted in favor of more promising treatment arms. Although both investigational treatments were rejected by the end of the trial, by using the adaptive design the investigators were able to reach the same conclusions with fewer patients exposed to risks (34 in the adaptive trial vs 75 if a frequentist design had been used).

One limitation of this design is that the analysis of some important prognostic covariates may not have sufficient statistical power due to the imbalance as a result of flexible assigning probability.

Adaptive Seamless Phase 2/3 Design

An adaptive seamless phase 2/3 design is a 2-stage design consisting of a learning phase (similar to a phase 2b trial) and a confirmatory phase (phase 3) in a single trial without suspending patient accrual. The term “seamless” indicates that there is no trial suspension between phase 2 and phase 3, with interim analyses for dose selection or futility stop after an initial dose-ranging trial. The decision of whether to stop early, continue to phase 2, or proceed to phase 3 may be made repeatedly rather than at a single decision at 1 time point. Adaptations at the interim analyses may involve different aspects, including treatment selection, sample size reassessment, and stopping for futility. A typical approach is to plan the power of a seamless phase 2/3 trial for the confirmatory phase and use data collected from the learning phase to estimate certainty about the treatment effect using a confidence interval approach. The data collected from both phases are used in the final analysis, which improves the efficiency because fewer patients are needed to obtain about the same information as could be gathered from separate traditional phase 2 and 3 trials. The overall type I error rate is controlled at a prespecified level regardless of the adaptation performed at interim. Such flexibility allows the use of Bayesian methods for interim analyses without affecting the frequentist significance level. Practical aspects concerning the planning and implementation of these adaptive trials have been discussed in the literature.^19,20

Bayesian Designs

A Bayesian framework that incorporates uncertainty into a parameter in a quantitative manner (ie, expressed by a prior distribution on the parameter) is becoming more common in adaptive trials. The term “adaptive” is not always synonymous with Bayesian, of course, but with more complex adaptive trial designs, many times a Bayesian approach is useful. Instead of estimating the likelihood that an observed response to the investigational treatment could have happened by chance with a frequentist approach, a Bayesian design calculates a predictive probability of treatment efficacy and makes inferences based on the currently available data. In other words, the Bayesian approach continuously learns about the treatment efficacy as data accumulate and updates the posterior probability. The Bayesian approach can be used in many types of adaptations. In dose-finding phase 1 trials, a Bayesian alternative to the 3+3 MTD finding is the Bayesian CRM design. A well-known Bayesian adaptive dose-response trial is Pfizer’s ASTIN trial in acute stroke.²¹ Some conservative hybrid designs also exist, with a short 3+3 phase to constrain the dose-response curve before switching to the full CRM design.²² The Bayesian adaptive seamless phase 2/3 design has also attracted great interest. For example, recently a seamless phase 2/3 oncology trial design was proposed to use Bayesian approach to select a sensitive patient population for the development of a targeted therapy.²³ In addition to these 2 most commonly applied fields, the Bayesian framework has also been used in outcome-adaptive randomization to model joint efficacy and toxicity outcomes or to select the best treatment for a subgroup of patients based on their prognostic factors with multicourse treatment strategies. Also, a new approach was proposed to reduce the required sample size for a group-sequential survival trial using the Bayesian adaptive model selection strategy.²⁴

As mentioned, one of the most attractive features of the Bayesian approach is its ability to incorporate historical data or prior knowledge into the design. It is very important that we carefully choose an appropriate prior distribution for parameters of interest. Some researchers choose noninformative priors to avoid creating a design too sensitive to the prior information. However, such a design does not take existing information into consideration, and noninformative priors may cause the design to be oversensitive to early results. One alternative is to weigh less on the prior by raising the term in the posterior density concerning the prior data to a power that is less than 1 so that the prior does not overly dominate inference, which is necessary when the prior is based on rather limited historical data.

To assess frequentist characteristics such as type I and type II error rates of a Bayesian design, an intensive simulation study that considers a wide range of possible scenarios should be carried out. The main purpose is to evaluate how robust and reliable the Bayesian design is under different circumstances, especially when compared with a standard frequentist design.

Example: Sample Size Reestimation Phase 2 Design With a Bayesian Framework

An adaptive phase 2 design using Bayesian predictive probability and Simon’s minimax design criterion was proposed by Lee and Liu in 2008.²⁵ The design method calculates a predictive probability based on the investigator’s prior knowledge of the treatment’s efficacy and information collected on the current patients at the treatment of each additional patient enrolled after treating the first 10 patients. After the first 10 patients, an interim decision will be made at any interim time, based on the estimated efficacy and toxicity of the treatment. The authors illustrated the design method in 2 examples. Assuming a beta-distributed prior distribution of the response rate and a binomial distribution of the observed number of responses, the predictive probability calculated from the beta-binomial posterior distribution is used to decide whether the trial should be stopped early for either efficacy or futility reasons. Given the same design parameters, type I and type II error rate constraints, and maximum sample size, based on a simulation study, the expected sample size required for the Bayesian design is smaller compared with Simon’s mini­max design. Although in certain situations the expected sample size of the Bayesian design is larger than Simon’s optimal design for a given maximum sample size, this design method provides opportunities to modify the number of stages and sample sizes, while allowing continuous monitoring of the outcome.

Into the Biomarker-Trial Era

In conventional medical care, the regimen is usually guided by standards of care based on results of large cohort studies. However, these large cohort studies do not take into account the genetic variability of individuals or subgroups within a population. Increasingly, investigators are asking if they can improve the outcome (eg, patient survival) by personalizing the regimen to treat each patient more effectively. Trials using dynamic treatment regimens can be viewed as a stream of personalized medicine designs by allowing treatment to vary with time based on individual prognostic factors^26,27 or ongoing individual response.^17,28

Recently, due to the rapid development of advanced technologies in a number of molecular profiling areas, including proteomic profiling, metabolic analysis, genetic testing, and high-throughput deep sequencing, biomarker-adaptive design has quickly evolved, allowing for a greater degree and broader range of personalized medicine. Biomarkers can be used as intermediate end points to identify target patient populations that are most sensitive to a particular treatment (ie, separate good- and poor-prognosis patients), to predict patients’ responses to a treatment, to replace the primary clinical outcome as a surrogate end point for early disease diagnosis, and to identify novel drug targets. These applications are the major contributors to the development of personalized medicine. Biomarker-adaptive design refers to a trial design where the adaption is made based on the response of 1 or more biomarkers (usually multiple) associated with the disease. Biomarkers have already been used in almost every stage of drug development, from compound discovery and preclinical studies through each phase of clinical trials and into postmarketing evaluations. One example is a 3-year phase 1/2 trial launched in 2009 for the treatment of metastatic colorectal cancer using proteomic profiles to predict those patients who will respond to the investigational drug imatinib mesylate. In practice, biomarker adaptation can be combined with other adaptation methods in a clinical trial. The selection of surrogate biomarkers in a clinical trial needs to be carefully justified to ensure the association between biomarkers and clinical outcomes under the investigational drug. Establishing a linkage between a biomarker test and the clinical end point is very important in guiding therapy decisions in personalized medicine, which is usually dependent on the ability to classify patients into distinct subgroups based on their genomic and/or proteomic profiles.

Conventional clinical trials not involving biomarker tests only estimate the average treatment effect in the overall target population. In order to evaluate biomarker-guided treatments, several biomarker designs have been proposed, such as biomarker-stratified designs, biomarker-strategy designs, and enrichment designs. In a biomarker-stratified design, patients are randomly assigned regardless of their biomarker status, but the analysis is stratified by the biomarker status. The biomarker-stratified design estimates without bias treatment effects across biomarker-defined subgroups by maximizing the benefits of randomization. However, in some cases it is believed that the treatment may benefit only patients of certain biomarker-defined subgroups; therefore, the use of a biomarker-stratified design may not be ethical. In this situation, the biomarker-enrichment design can be used as an alternative to estimate the treatment effect among patients of certain biomarker status. Initially, biomarker profiles are obtained from all participating patients but, based on these profiles, only patients who are believed to benefit from the treatment will remain in the trial. Another biomarker design is the biomarker-strategy design. In this design, patients are randomly assigned either to a biomarker-guided arm or to a control arm that is not biomarker based. Within the biomarker-guided arm, biomarker-positive patients are assigned to the investigational treatment, and biomarker-negative patients, together with patients in the control arm, are given the standard treatment. A drawback of the biomarker-strategy design is that the observed treatment effect may be diluted by having overlapping on-treatment assignment between the treatment arm and the control arm, consequently reducing the trial’s statistical power. Additionally, it may not be easy to interpret a significant observed treatment effect because it may be either that the biomarker is useful in guiding the personalized regimen or that 1 treatment is simply more effective than the other, regardless of the biomarker status.²⁹

Example: Adaptive Biomarker Trials

Advances in high-throughput genomic technologies offer opportunities to select sensitive patients in a clinical trial. Freidlin and Simon introduced an adaptive design that incorporates a gene expression classifier of drug sensitivity into a randomized phase 3 design as a possible second stage of the analysis.30 Freidlin, Jiang, and Simon published a follow-up paper in the same journal,³¹ providing a cross-validation extension of the adaptive signature design. Rather than dividing patients into training set and test set, the new version of the design uses a cross-validation method for gene signature development and testing and is therefore more efficient.

Lee and colleagues^32,33 applied an outcome-based adaptive randomization trial design in a 4-arm phase 2 trial that has drawn wide attention in recent years. The goal of the BATTLE (Biomarker-Integrated Approaches of Targeted Therapy for Lung Cancer Elimination) trial is to establish multiple-biomarker classifiers to guide the treatment of patients with non–small cell lung cancer. Patients were assigned to 1 of 4 biomarker groups based on their status of 4 biomarkers (epidermal growth factor receptor mutation/amplification, KRAS and/or BRAF mutation, vascular endothelial growth factor (VEGF) and/or VEGF receptor expression, and RXR and/or cyclin D1 expression). In the first stage of the trial, the initial group of patients was randomized to receive 1 of 4 treatments. Once sufficient treatment data had been collected, the trial moved to its second, adaptive stage. Based on their biomarker status, the remaining patients were assigned to the treatment arm most likely to result in the best outcome. By assigning patients to a treatment arm that is most likely to benefit them, the trial’s efficiency was increased.

Other Adaptations

In addition to the adaptation designs mentioned above, other adaptations have been proposed and/or applied in clinical trial design, including hypothesis-adaptive design, adaptive enrollment, and primary end point adaptive design. The common goal is to reduce the time and resources required or to explore a wider range of hypotheses (eg, more doses or treatment arms) and/or in a broader population (eg, more subgroups). In spite of the many advantages, there are certain concerns associated with these approaches, some of which are addressed in the Discussion section.

Discussion

Adaptive methods have rapidly emerged as powerful tools for clinical trial design. Researchers are employing adaptive trial design, hoping to improve efficiency and save resources. However, simple use of an adaptive design does not guarantee these benefits. They may be offset by bias, causing an increased overall type I error rate, particularly with an unblinded design. It should be remembered, however, that an adaptive interim analysis may require unblinding the data. Bias can arise from patient or dose selection, early withdrawal, treatment switching, modifications of eligibility criteria, change of end point, failure in establishing clinical relevance with the biomarker, and change of statistical analysis. For example, in a typical play-the-winner adaptive design, the decision made at interim review may not be optimal because a “winner’s” observed interim treatment effect, based on a relatively small sample size by chance, can be more favorable than it actually is. On the other hand, at interim review a drop-the-loser adaptive trial has the potential to inflate the type II error rate by leading investigators to make wrong decisions on which treatment arms/doses should be eliminated and which should be reserved. In other adaptive methods, reasons causing bias and the magnitude of such bias are not yet well understood. But, as a general rule, steps should be taken when possible to improve the power of the study and to avoid an inflated overall type I error rate.

Some types of adaptation may also result in a totally different trial from that originally planned, in which either the target population has been shifted or there is an inconsistency between the original hypotheses and the analysis actually performed. This can jeopardize the trial’s integrity and can cause difficulties in interpreting results regarding the original questions the trial was intended to answer. Because these problems may have a significant impact on the conclusion’s accuracy and reliability, it is important to consider potential sources of bias when planning and analyzing adaptive trials.

In some recently developed trials, typically trials for chronic disease, a biomarker or a surrogate end point is used as the end point or is used for the interim analysis to determine the adaptive modification. This strategy has the potential to foster advances in personalized medicine by identifying patient subgroups who are more likely to respond to a certain treatment based on their genetic profiles in addition to their clinical characteristics. If there is uncertainty regarding the biomarker’s predictive ability for the corresponding clinical end point, however, the use of a surrogate may lead to difficulties for both design and final assessment of treatments. Investigators need to provide analytical validation to ensure that the biomarker’s clinical relevance is reproducible (ie, significant statistical correlation between biomarker and clinical end point) and has acceptable levels of sensitivity and specificity. In other words, it is critical to ensure that a surrogate accurately reflects the true clinical end point.³⁴ Introduction of bias to the trial by using a biomarker surrogate is also a concern because of such uncertainty attached; therefore, statistical adjustments are required to control the type I error rate. Additionally, in many circumstances, especially in complex diseases such as cancer, not all treatment effects can be fully accounted for by a single biomarker. The use of multiple biomarkers for 1 disease, however, may lead to a more comprehensive assessment of treatment effects.

Conclusion

The discovery-to-market time of drug development has increased dramatically, but the number of approved new drug products is on the decline. Adaptive design has changed the conduct and implementation of clinical trials. Today’s adaptive designs are intended to shorten the discovery-to-market time without decreasing the information collected, to model and monitor the real-time effectiveness of treatment, and also to personalize treatment based on individual genetic profiles and molecular factors. The number of phase 1 and 2 trials that use an adaptive design has increased rapidly. Even though the development of adaptive designs was focused on phase 3 trials, as well as on seamless phase 2/3 trials, fewer phase 3 trials have employed adaptive designs. Adaptive trial design is more popular in early-stage studies because these designs are most helpful when there is little evidence about the treatment effect, a fact more likely to be true in phase 1 and 2 trials than in phase 3 trials. In addition, the FDA raised concerns about the potential bias resulting from unblinding the data of patient response, which is less likely with phase 1 and 2 trials than with phase 3 trials. Currently, for ethical and budgetary reasons, both researchers and the industry are moving toward generalizing adaptive designs to confirmatory phase 3 trials. In the next 5 years or so, adaptive designs undoubtedly will continue to draw more attention from the FDA, industry, and researchers and will most likely become more mainstream, especially for phase 3 trials. In the near future, the use of biomarker-defined subgroups in large-scale clinical trials will likely increase, particularly in light of advances in biological technologies and the increasing availability of bioinformatics software and computational power.

Wise use of adaptive trial design is crucial. In spite of their flexibility and efficiency, concerns remain about the integrity and validity of adaptive trials; challenges remain in the statistical analysis of complex adaptive trials. Both type I and type II error rates must be carefully controlled to account for the bias introduced by adaptation(s) and to ensure sufficient sample size to achieve the desired statistical power. In practice, the FDA recommends examining the consistency of treatment effects between study stages, as well as comparability between patients enrolled before and after the adaptation. As many types of adaptive designs are relatively new to most pharmaceutical and biotechnology companies, the potential exists that modifications based on interim analyses could have a negative effect on a trial that has not been properly monitored and executed. Researchers and industry need detailed regulatory guidelines and intensive discussion concerning the valid use of certain adaptive design methods as well as for development of appropriate statistical methods for analyzing complex adaptive trials.

Acknowledgments

The authors would like to thank Peggy Schuyler for editorial advice.

This research was supported in part by the Lung Cancer Special Program of Research Excellence (SPORE) (P50 CA090949), Breast Cancer SPORE (P50 CA098131), GI SPORE (P50 CA095103), and Cancer Center Support Grant (CCSG) (P30 CA068485).

References

1. Cornfield J, Halperin M, Greenhouse SW. An adaptive procedure for sequential clinical trials. J Am Stat Assoc. 1969;64:759-770.
2. Robbins H. Some aspects of the sequential design of experiments. Bull Am Math Society. 1952;58:527-535.
3. Zelen M. Play the winner rule and the controlled clinical trials. J Am Stat Assoc. 1969;64:131-146.
4. Wei LJ, Durham S. The randomized play-the-winner rule in medical trials. J Am Stat Assoc. 1978;73:840-843.
5. O’Quigley J, Pepe M, Fisher L. Continual reassessment method: a practical design for phase 1 clinical trials in cancer. Biometrics. 1990;46:33-48.
6. O’Quigley J, Shen LZ. Continual reassessment method: a likelihood approach. Biometrics. 1996;52:673-684.
7. Chang M, Chow SC. Power and sample size for dose response studies. In: Ting N, ed. Dose Finding in Drug Development. New York, NY: Springer; 2006.
8. Flaherty KT, Puzanov I, Kim KB, et al. Inhibition of mutated, activated BRAF in metastatic melanoma. N Engl J Med. 2010;363:809-819.
9. Berry SM, Spinelli W, Littman GS, et al. A Bayesian dose-finding trial with adaptive dose expansion to flexibly assess efficacy and safety of an investigational drug. Clin Trials. 2010;7:121-135.
10. Korn EL, Freidlin B. Outcome-adaptive randomization: is it useful? J Clin Oncol. 2011;29:771-776.
11. Berry DA. Adaptive clinical trials: the promise and the caution. J Clin Oncol. 2011;29:606-609.
12. Burington BE, Emerson SS. Flexible implementations of group sequential stopping rules using constrained boundaries. Biometrics. 2003;59:770-777.
13. Hughes MD, Freedman LS, Pocock SJ. The impact of stopping rules on heterogeneity of results in overviews of clinical trials. Biometrics. 1992;48:41-53.
14. Jennison C, Turnbull BW. Sequential equivalence testing and repeated confidence intervals, with applications to normal and binary responses. Biometrics. 1993;49:31-43.
15. Lehmacher W, Wassmer G. Adaptive sample size calculations in group sequential trials. Biometrics. 1999;55:1286-1290.
16. Bretz F, Schmidli H, Konig F, et al. Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: general concepts. Biom J. 2006;48:623-634.
17. Murphy SA. An experimental design for the development of adaptive treatment strategies. Stat Med. 2005;24:1455-1481.
18. Giles FJ, Faderl S, Thomas DA, et al. Randomized phase I/II study of troxacitabine combined with cytarabine, idarubicin, or topotecan in patients with refractory myeloid leukemias. J Clin Oncol. 2003;21:1050-1056.
19. Jennison C, Turnbull BW. Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: opportunities and limitations. Biom J. 2006;48:650-655.
20. Emerson SS. Issues in the use of adaptive clinical trial designs. Stat Med. 2006;25:3270-3296.
21. Grieve AP, Krams M. ASTIN: a Bayesian adaptive dose-response trial in acute stroke. Clin Trials. 2005;2:340-351.
22. Yin G, Li Y, Ji Y. Bayesian dose-finding in phase I/II clinical trials using toxicity and efficacy odds ratios. Biometrics. 2006;62:777-784.
23. Brannath W, Zuber E, Branson M, et al. Confirmatory adaptive designs with Bayesian decision tools for a targeted therapy in oncology. Stat Med. 2009;28:1445-1463.
24. Wathen JK, Thall PF. Bayesian adaptive model selection for optimizing group sequential clinical trials. Stat Med. 2008;27:5586-5604.
25. Lee JJ, Liu DD. A predictive probability design for phase II cancer clinical trials. Clin Trials. 2008;5:93-106.
26. Thall PF, Sung HG, Estey EH. Selecting therapeutic strategies based on efficacy and death in multicourse clinical trials. J Am Stat Assoc. 2002; 97:29-39.
27. Ratain MJ, Mick R, Janisch L, et al. Individualized dosing of amonafide based on a pharmacodynamic model incorporating acetylator phenotype and gender. Pharmacogenetics. 1996;6:93-101.
28. Lavori PW, Dawson R. Dynamic treatment regimes: practical design considerations. Clin Trials. 2004;1:9-20.
29. Freidlin B, McShane LM, Korn EL. Randomized clinical trials with biomarkers: design issues. J Natl Cancer Inst. 2010;102:152-160.
30. Freidlin B, Simon R. Adaptive signature design: an adaptive clinical trial design for generating and prospectively testing a gene expression signature for sensitive patients. Clin Cancer Res. 2005;11:7872-7878.
31. 31. Freidlin B, Jiang W, Simon R. The cross-validated adaptive signature design. Clin Cancer Res. 2010;16:691-698.
32. Lee JJ, Xuemin G, Suyu L. Bayesian adaptive randomization designs for targeted agent development. Clin Trials. 2010;7:584-596.
33. Zhou X, Liu S, Kim ES, et al. Bayesian adaptive design for targeted therapy development in lung cancer – a step toward personalized medicine. Clin Trials. 2008;5:181-193.
34. Prentice RL. Surrogate and mediating endpoints: current status and future directions. J Natl Cancer Inst. 2009;101:216-217.