It is by no means straightforward to analyse the change in the rate of chronic obstructive pulmonary disease (COPD) exacerbations in clinical trials. Exacerbation rates do not follow a normal distribution, nor do they occur at random. High exacerbation rates in a few patients can make average rates difficult to calculate and interpret. So, surely, transforming exacerbation rates into numbers needed to treat (NNT) should help. Not necessarily so—this is the message from Professor Suissa's paper.1 He points out that the simplistic transformation from annual exacerbation rates to NNT in some published papers is misleading. He then goes on to present an alternative way of calculating NNT from survival curves showing time to first exacerbation, and a model to estimate such curves even if they are not presented.

I have used the exponential model suggested by Suissa, with the data from two of the arms of the Towards a Revolution in Chronic obstructive pulmonary disease Health (TORCH) trial2 to show how this works in practice. I chose the arms in TORCH, that compared combination fluticasone/salmeterol therapy with salmeterol alone, as this seems to me to be a fair way to estimate the impact of additional inhaled corticosteroids. Figure 1 shows pairs of modelled survival curves for pneumonia (in the upper part), and for COPD exacerbation (in the lower part). The NNT for each condition is derived from the vertical distance between the two curves, and is calculated as the inverse of this distance at any particular time point.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Survival curves for pneumonia and chronic obstructive pulmonary disease exacerbation in TORCH: fluticasone/salmeterol versus salmeterol (using Poisson/exponential model). This figure is only reproduced in colour in the online version.

The figure shows that the two sets of survival curves have quite different shapes. For pneumonia, the slope is shallow, and the gap between combination treatment and salmeterol widens steadily over 3 years. This means that the NNT(H) for one additional patient to suffer pneumonia on inhaled corticosteroids gets steadily smaller over time. By contrast, the lower two survival curves for time to first exacerbation are much steeper. Although the gap between the two curves widens initially, it shrinks again towards the end of the 3-year period. So, the NNT(B) for one additional patient to be free from pneumonia initially gets smaller, but then rises again at the end of the study period.

The patterns shown in the figure using the model explain why the NNT results in table 2 of his paper appear to favour the impact of inhaled corticosteroids on preventing COPD exacerbations after 1  year (in the Ferguson et al3 and Anzueto et al4 trials), but reverse to suggest a greater impact on the increased occurrence of pneumonia at 3 years (in TORCH).

As Suissa points out, all calculations of NNT are time-dependent. We need, therefore, to consider how the NNT changes over time, and as the survival curves in figure 1 demonstrate, this is not always intuitively obvious. There is a further problem with survival curves of the time to first event for events that occur frequently. Patients are only counted up until their first exacerbation, and any subsequent exacerbations in the same patient are not considered. For this reason, using NNT will always be problematic when making comparisons between rare and frequent events.

It is time to abandon any attempt to calculate NNT to prevent a single ‘event’ in trial reports and stick to NNT for one participant to be event-free, as shown in the survival curves. The concept of ‘event-based’ NNT is fundamentally flawed, as Aaron has already pointed out.5 Given that it is not possible to treat less than a whole patient, NNT must be calculated using the patient, rather than an event, as the unit of analysis. The survival curves of time to first exacerbation and reporting of exacerbation rates provide much more information, and should be published in trial reports in preference to a single NNT. This would allow readers to make their own assessments.

A summary of the evidence from all trials comparing combination therapy with long-acting β₂-agonist (LABA) alone is available in a Cochrane review.6 The review contains a summary of findings table which presents an overview of the data from all identified randomised trials. The overall impact of an additional inhaled corticosteroid on exacerbations is presented in two ways. Given an average rate of one exacerbation per year on LABA in the trials, the expected rate on combination therapy would be 0.76 exacerbations per year (95% CI 0.68 to 0.84). However, the review points out that there was considerable heterogeneity between the rate ratios in the included trials, and that high withdrawal rates limit our confidence in these results. Over a year, the review found that 47 people per 100 suffered one or more exacerbations on LABA; on average, this fell to 42 per 100 (95% CI 38 to 46) on combination therapy. The number suffering an episode of pneumonia rose from three per 100 on LABA, to four per 100 (95% CI 3 to 5) on combination therapy over the same period. So, for patients who suffer frequent exacerbations, we would expect more exacerbations to be prevented per year than the number of additional episodes of pneumonia.

However, the debate about the risk and benefit of inhaled corticosteroids is not just about numbers. Exacerbations of COPD can be defined in numerous different ways, and so for that matter can episodes of pneumonia. This makes indirect comparisons between trials using different inhaled corticosteroids hard to interpret. Differences in risk or benefit between doses or products should certainly not be naively compared using NNTs, nor by comparing the p values for each dose or product. The correct statistical approach to making such comparisons is using a test for interaction.7 In the case of the increased risk of pneumonia in the Cochrane review, this test did not show a significant difference between the subgroups of trials on fluticasone and those on budesonide, or between higher and lower doses.6 This does not mean that there is ‘no difference’ between the risks or benefits on different doses of budesonide and fluticasone. It does mean that we are simply uncertain about the pair of survival curves for each dose and product, and how much they vary one from another.