Statistics from Altmetric.com
The Study to Understand Mortality and Morbidity in COPD (SUMMIT) is a randomised controlled trial conducted in patients with COPD and comorbid cardiovascular disease, designed to assess the ‘impact of Fluticasone Furoate/Vilanterol combination (FF/VI) and its individual components on the survival of patients with moderate COPD and either a history of CVD or at increased risk for CVD’.1 Fluticasone furoate is an inhaled corticosteroid (ICS) and vilanterol is a long-acting β-agonist (LABA). The trial is enrolling 16 000 such patients with moderate COPD randomly assigned to once daily treatment with this ICS–LABA combination, the ICS only, the LABA only or placebo. Patients in this event-driven trial will be followed up until 1000 deaths occur, which is expected to take up to 44 months. Mortality is the primary endpoint and the primary comparison is between the ICS–LABA combination and placebo. The study is designed as a superiority trial with 90% power to detect a 30% reduction in all-cause mortality when comparing the ICS–LABA with placebo at the two-sided 1% significance level. Secondary endpoints will also be studied, including lung function decline and composite cardiovascular events.
The design of SUMMIT is similar to that of the Towards a Revolution in COPD Health (TORCH) trial,2 with the exception that SUMMIT focuses on a study population of patients with moderate COPD and at cardiovascular risk. As well, while all patients in TORCH were followed for an exact 3-year period, SUMMIT being event-driven is expected to follow-up patients between 15 and 44 months.
At first glance, the SUMMIT trial appears to have set unattainable goals. Indeed, the magnitude of the targeted effect in terms of power, a 30% reduction in all-cause mortality (HR=0.70), seems overly ambitious. In the TORCH trial, the target for this same outcome was a 27% reduction (HR=0.73) with an even smaller sample size. Moreover the primary outcome is all-cause mortality, rather than cause-specific mortality. Because of the nature of the study population, one would expect that respiratory or cardiovascular mortality be the focus of the outcome, particularly in view of a rationale for this trial, namely, that ‘combination treatment seemed to have a similar effect on respiratory and cardiovascular mortality in the primary TORCH report’.1 ,3 Including non-respiratory and non-cardiovascular deaths in the outcome can be thus perceived as potentially diluting any beneficial effects of the treatment. In all, it can be construed that SUMMIT set the bar much too high to find significant effectiveness for the drug and, if a benefit is indeed reached with this trial, it will be against all odds. Not necessarily so. In fact, several methodological aspects of the study design could affect the results in the other direction.
The design of the TORCH trial revealed several concerns that could have led to biased estimates of the effects of the ICS–LABA combination inhaler on mortality and on the secondary outcomes.4–8 Many of these methodological limitations are still present in the design of the SUMMIT trial and could also affect the estimates of this trial. In this paper, we review these methodological issues in the context of the SUMMIT trial and discuss how they could potentially affect the results of this trial and the interpretation of the data.
The placebo comparison group
One of the major criticisms with the TORCH trial is the choice of ‘placebo’ as the comparison group, which was incompatible with the study objective of ‘usual care’ as the comparison group. Indeed, over 59% of the patients in the placebo comparison group were on maintenance therapy (LABA, ICS or a long-acting anticholinergic) when they entered the study, but these treatments were discontinued at randomisation and replaced by a placebo for the 3-year follow-up. It was in fact somewhat surprising that the TORCH study results did not actually show inordinate benefits in view of the expected advantage from discontinuing the maintenance therapy of patients who visibly must have needed it and replaced it with placebo for 3 years.8
The SUMMIT trial is clear about the study question, aiming to ‘test the hypothesis that treatment with combined inhaled corticosteroids and long-acting β-agonists will reduce mortality when compared with placebo’.1 However, here again, the patients in the placebo comparison group in SUMMIT who were on maintenance therapy when they entered the study will also have them discontinued at randomisation and replaced by a placebo during follow-up. One is justified to question whether using such a comparison group is actually appropriate and whether it does not provide an inherent advantage to the treated group by giving placebo for years in replacement for pretrial maintenance therapy. These effects are presented below.
Effect of treatment discontinuation at randomisation
When study patients entering the trial must terminate their maintenance treatment at the time of randomisation, such discontinuation becomes an inherent component of the intervention when the discontinued drug is of the same class as the study drug.
This phenomenon was illustrated using data from the OPTIMAL trial that assessed the effect of adding ICS to long-acting bronchodilators in COPD patients on the incidence of exacerbation over 1 year follow-up.9 A reanalysis showed that the overall HR of exacerbation associated with ICS relative to bronchodilators was estimated as 0.79 when all subjects were combined. However, among the patients who had previously used ICS and had to discontinue, the HR was 0.71, while for those who had not used ICS and thus did not have to discontinue, it was 1.11.5 In essence, this phenomenon warns that an overall effect estimate in such a trial is comprised of two different components: the effect of discontinuing the drug among the previously treated who had to discontinue and the effect of initiating the drug among the treatment-naive users. This effect was present in the TORCH trial where over 60% of the placebo patients had to suddenly discontinue their maintenance therapy and use placebo for 3 years.
In the SUMMIT trial, three treatments are discontinued and studied, namely, ICS, LABA and the ICS–LABA combination. Although recruited patients are those that ‘the practitioner believes can be adequately managed without these medications’,1 it remains that these medications are used by over 70% of COPD patients, including the moderate ones targeted in this trial, as suggested by market data and several trials.7 ,10
In the context of the SUMMIT trial, we illustrate this phenomenon focusing specifically on the primary comparison between the ICS–LABA combination and placebo. Figure 1 depicts three different sources of discontinuation. First is the subgroup of patients who will be on an ICS–LABA combination before randomisation and who will be assigned to either the study ICS–LABA combination or placebo. This contrast will clearly provide a direct effect of the discontinuation of the combination drug, not its initiation. Figure 1 also shows that the patients who stop their ICS at randomisation and are assigned to the study ICS–LABA combination or placebo during follow-up will contribute a double effect, namely, the effect of ICS discontinuation and the effect of initiating LABA. Similarly, the subjects who were on LABA will contribute to the effect of LABA discontinuation and the effect of initiating ICS.
Figure 2 depicts the valid comparisons related to the desired effects of initiation of the ICS–LABA combination and its components. It shows that it is only among the patients who are treatment-naive of any ICS or LABA that the ICS–LABA combination can be directly compared with placebo to provide a valid estimate of the effect of the treatment.
Therefore, these depictions show that the computation of a single estimate of the effect based on comparing ICS–LABA with placebo across all four scenarios will encompass a mix of unspecific estimates of the effects of various drug discontinuations and initiations, clearly dissonant with the sought after effect of ICS–LABA.
Censoring at treatment discontinuation in follow-up
One of the important methodological contributions of the TORCH trial has been that it was among the first trials in COPD to provide a valid intent-to-treat analysis by collecting data on death, the primary outcome, after study (and treatment) discontinuation during follow-up, untainted by the reasons for discontinuing. The censoring of follow-up for such patients is unlikely to be random and thus, faced with such informative censoring, an on-treatment analysis can be biased. Indeed, a reanalysis of the TORCH data showed that withdrawals occurred earlier and were more likely in the placebo group (44%) compared with the ICS–LABA combination group (34%), as were patients previously on ICS and with a history of multiple exacerbations.11
Thus, in the SUMMIT trial, mortality will not be affected by such censoring as all deaths occurring during follow-up will be identified, irrespective of whether the patients discontinued treatment. However, such is not the case for secondary outcomes such as lung function decline and exacerbations, which are not measured after that point. Such data truncation can lead to bias if the reasons for discontinuation are associated with the outcome and differ between treatments, which can affect the exacerbation rates.12 It will also introduce bias in computing the rate of decline in lung function over time, as it is affected by the phenomenon of ‘regression to the mean’ when follow-up is discontinued.6
Thus, the censoring of follow-up at treatment discontinuation in SUMMIT will not affect mortality but could exaggerate the effects for the secondary outcomes such as lung function decline and exacerbations.
Factorial design analysis
Like TORCH, the SUMMIT trial is structured as a 2×2 factorial design of an ICS (yes/no) and a LABA (yes/no). However, despite generating data on four exposure groups, the study intends to compare only the ICS–LABA group with placebo, thus squandering much needed power. Moreover, a full 2×2 factorial analysis would quantify the independent contribution of each drug component on the outcome, which would provide valuable scientific information about these drugs. For the TORCH trial, for example, such an analysis showed that the LABA component of the combination was associated with a significant 17% reduction in mortality (rate ratio 0.83; p=0.0043), while the ICS component provided no reduction (rate ratio 1.00; p=0.99).4 ,5
The conduct of real-life mega-trials of the effects of already marketed drugs on major outcomes has been strongly advocated.13 In COPD, mortality and exacerbations requiring hospitalisation are the major outcomes that deserve the attention of such trials.14 In fact, COPD has seen many mega-trials, including TORCH, as well as the Understanding Potential Long-Term Impacts on Function with Tiotropium (UPLIFT) and TIOSPIR trials of tiotropium.2 ,15 ,16 As such, the SUMMIT mega-trial is a commendable initiative to study the effects of COPD treatment on mortality.1
Nonetheless, while the randomised trial is the fundamental design to evaluate drug effectiveness, it is not immune from methodological vulnerabilities that can distort its results. The mere act of randomisation is not sufficient. We showed that the SUMMIT trial will have several key methodological challenges to surmount, including the dubiousness of the placebo comparison group and the inherent effects of drug discontinuations at recruitment. As a result, SUMMIT will estimate an HR comparing ICS–LABA with placebo, but this single value will in fact conceal a mix of diverse estimates of the effects of drug discontinuations and initiations. Consequently, the result of this mega-trial may not, after all, yield a valid estimate of the sought-after impact of this ICS–LABA combination treatment that can be translated into evidence-based clinical practice in the management of patients with COPD.
Competing interests SS has participated in advisory board meetings or as speaker in conferences for AstraZeneca, Boehringer-Ingelheim, GlaxoSmithKline, Novartis, Nycomed and Pfizer, and has received research funding from Boehringer-Ingelheim, GlaxoSmithKline and Schering-Plough.
Provenance and peer review Not commissioned; internally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.