Article Text
Abstract
Background Several recent trials in chronic obstructive pulmonary disease (COPD) have assessed the effectiveness of the fluticasone–salmeterol combination inhaler in preventing COPD exacerbations, while finding an increased risk of pneumonia. The number needed to treat (NNT) is a simple measure to perform the comparative benefit–risk impact, but its calculation involving repeated outcome events such as COPD exacerbations has been incorrect. We describe the proper methods to calculate the NNT and, using data from published trials, apply them to evaluate the relative impact of fluticasone–salmeterol treatment on exacerbations and pneumonias in patients with COPD.
Methods We review the fundamental definition of NNT and quantify it for situations with varying followup times. We review the ‘eventbased’ NNT, proposed and used for repeated event outcomes, show its inaccuracy, describe its proper use and provide an approximate formula for its application.
Results We show that a 1year trial of the fluticasone–salmeterol combination versus salmeterol used the incorrect eventbased approach to calculate the NNT as two patients that need to be treated for 1 year to prevent one COPD exacerbation, when the proper calculation results in a NNT of 14. In contrast, 20 patients need to be treated to induce one pneumonia case. For the TORCH trial, the NNT is 44 patients treated for 3 years with fluticasone–salmeterol versus salmeterol to prevent one exacerbation compared with 16 patients to induce one pneumonia case.
Conclusions The NNT is a useful measure of the effect of drugs, but its proper calculation is essential to prevent misleading clinical practice guidelines.
Statistics from Altmetric.com
Key messages
What is the key question?

The number needed to treat (NNT) is a simple measure to assess the benefitrisk impact of a drug, but its calculation involving repeated outcome events such as chronic obstructive pulmonary disease (COPD) exacerbations has been inaccurate in several trials of the fluticasone–salmeterol combination.
What is the bottom line?

When properly computed, the NNT measure shows that pneumonias are just as important as exacerbations in the benefit–risk assessment of the fluticasone–salmeterol combination in COPD.
Why read on?

The correct computational approaches for the NNT described in the paper illustrate the discrepancies: the NNT for the fluticasone–salmeterol combination versus salmeterol in a recent trial is in fact 14 patients who need to be treated for 1 year to prevent one COPD exacerbation, not the reported 2 patients estimated by the inaccurate approach.
Introduction
The TORCH (Towards a Revolution in Chronic obstructive pulmonary disease (COPD) Health) randomised controlled trial was the first to identify the increased risk of pneumonia associated with inhaled corticosteroid use in patients with COPD.1 It reported that patients receiving fluticasone propionate combined or not combined with salmeterol had a significant 52% increase (HR 1.52; 95% CI 1.32 to 1.76) in the incidence of pneumonia adverse events during the 3year followup compared with patients receiving placebo or salmeterol alone.2 Subsequently several other randomised trials and metaanalyses confirmed this increase in risk.3–8
An issue that has been raised regarding this increase in the risk of pneumonia is its relative importance visàvis the prevention of COPD exacerbations. The Investigating New Standards for Prophylaxis in Reducing Exacerbations (INSPIRE) trial, which compared the fluticasone–salmeterol combination with tiotropium in a 2year trial conducted an extensive investigation of pneumonia adverse events.4 ,9 It identified 87 pneumonia reports from adverse event records compared with 2255 COPD exacerbations, concluding that ‘pneumonia is much less frequent than exacerbation in COPD’. This conclusion can be misunderstood to imply that the adverse impact of fluticasone on pneumonia is much less important than its benefit on COPD exacerbations.
Using the relative frequency of two outcomes, such as exacerbations and pneumonias, to judge the relative importance of the risk and benefit of a drug can be misleading since it does not account for the actual effect of the drug. Instead, the number needed to treat (NNT) is a simple and valid tool that permits such a comparative impact to be assessed.10 This measure has been advocated and used in several recent studies of treatment for COPD.11 However, its calculation when dealing with outcomes involving repeated events such as exacerbations has been criticised, leading to possibly distorted estimates of the NNT.12–14
In this paper, we review these methods to calculate the NNT and provide formulae for the proper techniques. We also apply these methods to assess the relative impact of fluticasone–salmeterol on exacerbations and pneumonias in patients with COPD using data from several recently published randomised trials.
The NNT measure
The NNT to assess the effectiveness of a drug treatment is measured from the difference between two groups of patients in the cumulative incidence of the outcome (CI) over a fixed followup time period. This difference CI_{0}−CI_{1}, where 0 represents the reference treatment or placebo and 1 the treatment under study, represents the proportion of patients for whom the outcome was prevented due to the drug. Inverting this difference (1/(CI_{0}−CI_{1})) will produce the number of patients that need to be treated by the drug to prevent one patient with the outcome, in other words the NNT.10 For example, a drug trial that produces an incidence of the outcome after 1 year of 1/100 (0.01) in the drugtreated group compared with 3/100 (0.03) in the placebo group implies that the drug prevents two outcome events per 100 treated for 1 year, corresponding to an NNT of 50 (1/0.02). That is, 50 patients need to be treated continuously for 1 year to prevent one patient incurring the outcome event. The NNT is now used extensively in randomised trials and observational studies to provide an additional and userfriendly measure of the impact of a drug or treatment on a given disease outcome.10 ,15
While the calculation of the NNT as shown is arithmetically simple, this is only so when trials follow all patients up for the same time and observe a single outcome event per patient. In practice, however, most trials involve unequal followup times while some will study recurrent outcomes with multiple events, such as COPD exacerbations. The NNT calculation is then not as straightforward in these situations.12–14
Unequal followup and single event: patientbased NNT
When followup times vary between patients, the CI of an outcome event cannot be calculated as a proportion of subjects. It must instead be based on the Kaplan–Meier approach, which estimates the CI accounting for variable followup.16 The NNT can then be directly computed by inverting the difference in the CI of the outcome between the two groups at the desired time point of followup from the Kaplan–Meier curves, or alternatively, approximated from the HR.17 It will estimate the number of patients that need to be treated by the drug to prevent one patient with the outcome over the given followup time. An example is the Kaplan–Meier curve provided for time to pneumonia in the INSPIRE trial.9 From the curves, the 2year CI of pneumonia is 0.094 and 0.049 for fluticasone–salmeterol combination and tiotropium respectively for NNT=1/(0.094–0.049)=22 patients for 2 years to induce a pneumonia. The curves also permit the NNT to be computed at any other time point during followup, for example at 1 year, NNT=1/(0.06–0.03)=34 patients for 1 year to induce a pneumonia.
Unequal followup and multiple events: eventbased NNT
With outcomes involving an event that can recur multiple times during followup, such as COPD exacerbations, most trials use the incidence rate (IR) to quantify the frequency. It is computed as the total number of events divided by the total amount of persontime, to account for varying followup times. The effect of the drug treatment is then measured by the difference in IRs between two groups, namely IR_{0}−IR_{1}, where 0 represents the reference treatment or placebo and 1 the treatment under study. This difference represents the IR of prevented events per personmoment. Several studies have, in this case, used an alternative formula to compute the corresponding NNT as 1/(IR_{0}−IR_{1}). This NNT, based not on the CI but on the IR, was interpreted as the NNT to prevent one ‘event’ over a given time period.11
An example is the trial by Anzueto that randomised close to 800 patients with COPD to either the fluticasone–salmeterol combination or salmeterol alone, and followed them for 52 weeks.6 With varying followup times, the IR of moderate or severe exacerbations, which could occur more than once during a patient's followup, was 1.59 per patientyear in the salmeterol group and 1.10 in the fluticasone–salmeterol group. The authors thus computed the NNT as 1/(1.59–1.10)=2.04 rounded to 2, which was interpreted as ‘two subjects need to be treated for 52 weeks … in order to prevent one exacerbation per year’.
This alternative formula, called the eventbased NNT, has been criticised since it is not based on patients but on persontime, making the interpretation inappropriate.12 Indeed, in an example devised by Aaron and Fergusson, this approach based on rates can produce illogical results such as 0.5 subjects need to be treated for 1 year to prevent one exacerbation. Clearly, treating a half of a patient appears nonsensical. Of course, the rates can be modified and the NNT interpreted as ‘1 subject needs to be treated for 1 year to prevent 2 exacerbations’, but then we are deviating from the meaning of the NNT, which relates to preventing one patient with the event.
Proper use of the eventbased NNT
Because the eventbased NNT calculation is based on the IR and not the CI, its use and interpretation must be consistent with the definition of a ‘rate’. It is fundamental to understand that the rate is an ‘instantaneous’ measure of the incidence of an event. Thus, even though the Anzueto trial described above reported IRs of COPD exacerbation of 1.10 and 1.59 per patientyear, its proper usage should be instantaneous. To accomplish this, one should use the smallest measurable time unit in such studies, such as the day, as the instantaneous measure of time. The fact that a patient will not have two exacerbations on the same day makes the day a sufficiently small time unit. Thus, the more proper presentation of the rates of 1.10 and 1.59 per patientyear should be 0.0030 and 0.0044 exacerbations per patientday respectively. Of course, in studies with much shorter followup, such as studies of inpatients, a smaller time unit such as the hour can be used depending on the outcome.
By using the instantaneous version of the rate with the smallest measurable time unit, it is then appropriate to use this alternative formula for the NNT. However, this NNT can only be interpretable over the instantdefining time interval. For example, the TORCH study reported IRs of COPD exacerbation of 1.13 and 0.85 per patientyear in the placebo and fluticasone–salmeterol combination group respectively. The NNT was computed as 1/(1.13–0.85)=3.6 rounded to 4 and inaccurately interpreted as ‘NNT of four to prevent one exacerbation in 1 year’.1 Instead, the corresponding instantaneous rates are 0.0031 and 0.0023 per patientday corresponding to an NNT of 1250 patients treated with fluticasone–salmeterol on a given day to prevent one exacerbation on that day. Table 1 shows the proper instantaneous eventbased NNT for the three trials of fluticasone–salmeterol that computed the NNT using the yearly eventbased NNT.
It is important to note that the instantaneous eventbased NNT cannot be converted directly to a longer term NNT. Thus, for example, it is incorrect to take the NNT of 1250 patients treated with fluticasone–salmeterol on a given day to prevent one exacerbation from the TORCH trial, divide it by 30 and claim that the resulting value of 42 corresponds to the NNT for 1 continuous month of treatment to prevent one exacerbation. The next section provides formulae that permit the conversion to different time periods.
Converting eventbased to patientbased NNT
As noted previously, the fundamental flaw with the eventbased NNT as it is commonly used in many studies is that it is not based on the instantaneous rates, but rather on rates that have been extrapolated to a longer time span. It is simply incorrect to convert directly the eventbased NNT to a patientbased NNT. However, by first converting the instantaneous rate into a CI function over time, it becomes possible to then compute the corresponding patientbased NNT. To do this, an approximation based on the relation between the Poisson and exponential distributions can be used with the following formula: where CI is the cumulative incidence of the outcome event up to time t and IR is the incidence rate of outcome events measured in the same time units as t.14 Note that the CI and IR are practically equal when the IR is rare over the desired time span (IR less than 0.10 or even 0.15 per patient), making this conversion unnecessary. However, with more common outcomes such as COPD exacerbations (eg, IR=1.13 per patient per year in the TORCH study's placebo group), this formula becomes essential in the absence of the preferred Kaplan–Meier CI. It is also important to note that this formula is only approximate as it depends on the strong assumption that the events follow a Poisson distribution, which is not often the case. In fact, we know that this is clearly not so for COPD exacerbations as they tend to cluster in time and become more frequent.18–20 In this case, the formula will provide somewhat biased estimates of the NNT, which can only be properly computed from the Kaplan–Meier CI estimates.
In the example of the Anzueto trial, the authors used the rates of exacerbations (per patient per year) of 1.10 and 1.59 for the fluticasone–salmeterol and salmeterol groups respectively, leading to a reported ‘NNT’ of two as noted above. Instead, the conversion of the daily instantaneous rates of 0.0030 and 0.0044 per patient into CIs would give corresponding 1year CIs of 0.67 and 0.80, leading to a NNT of eight patients who need to be treated for 1 year to prevent one exacerbation. In fact, the paper actually provided Kaplan–Meir curves for the time to the first exacerbation, with 1year CI estimates of 0.60 and 0.67 for the fluticasone–salmeterol and salmeterol groups respectively, corresponding to a NNT of 14 patients needing to be treated for 1 year to prevent one exacerbation. This study illustrates well the inappropriateness of the eventbased NNT (2 instead of 14) and to a lesser extent the bias from the deviation from the Poisson assumption needed to convert the rate to the CI, giving an NNT of 8 instead of 14.
COPD exacerbations versus pneumonias
Table 2 displays the properly computed or approximated NNTs for five trials assessing the fluticasone–salmeterol combination against a longacting bronchodilator. The NNT to prevent one COPD exacerbation and to induce one pneumonia case are presented side by side. While some studies suggest that around twice as many patients need to be treated to induce a pneumonia compared with preventing a COPD exacerbation, the longer term TORCH and INSPIRE trials suggest an opposite benefit–risk impact.
Conclusion
The NNT is a simple measure of the effect of a drug or treatment that is often incorporated in reporting study results. Its calculation, however, is not straightforward when studies involve recurrent outcomes with multiple events. Several drug trials in COPD have miscalculated the NNT to prevent one exacerbation. For example, a 1year trial of the fluticasone–salmeterol combination versus salmeterol alone calculated the NNT as 2, when the proper calculation resulted in a NNT of 14 patients who need to be treated for 1 year to prevent one COPD exacerbation. Such miscalculations can have a major influence on population calculations or economic impact studies: an incorrectly calculated NNT of 2 patients instead of 14 would imply the treatment of 20 000 instead of 140 000 patients to prevent 10 000 exacerbations in any such study, which can seriously distort any populationbased cost analysis.
We showed that eventbased NNTs are inherently flawed and should simply never be used, except when based on the smallest time unit, such as the day rather than the year, which in itself is hardly ever relevant. While a formula was provided to convert eventbased to patientbased NNT, it is only approximate. In essence, the Kaplan–Meier CI curve is the most accurate technique to estimate the NNT in any trial with variable followup times.
An important application that we presented for the NNT is in assessing the relative importance of the benefit and the risk of a drug treatment. We showed the NNT provides a simple tool to compare the benefit of the fluticasone–salmeterol combination in terms of preventing COPD exacerbations versus its risk in inducing pneumonias. The NNT avoids misleading conclusions based on the frequency of these events, such as in the 2year INSPIRE trial, which compared the fluticasone–salmeterol combination with tiotropium, and found that the frequency of pneumonias (87 events) was much lower than the 2255 COPD exacerbations, suggesting a less important risk than benefit for fluticasone propionate. Instead, the NNT provides a more accurate and objective contrast of this benefit–risk assessment, with a quite different conclusion.
In all, the NNT is a useful measure of the effect of drugs, but its proper calculation is essential to avoid misleading clinical and public health decisions.
References
Footnotes

Competing interests The author has received research grants and/or participated in advisory meetings or as conference speaker for AstraZeneca, BoehringerIngelheim, Forest, GlaxoSmithKline, Merck, Novartis, Nycomed and Pfizer.

Provenance and peer review Not commissioned; externally peer reviewed.
Request permissions
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.