The NNT measure

The NNT to assess the effectiveness of a drug treatment is measured from the difference between two groups of patients in the cumulative incidence of the outcome (CI) over a fixed follow-up time period. This difference CI₀−CI₁, where 0 represents the reference treatment or placebo and 1 the treatment under study, represents the proportion of patients for whom the outcome was prevented due to the drug. Inverting this difference (1/(CI₀−CI₁)) will produce the number of patients that need to be treated by the drug to prevent one patient with the outcome, in other words the NNT.10 For example, a drug trial that produces an incidence of the outcome after 1 year of 1/100 (0.01) in the drug-treated group compared with 3/100 (0.03) in the placebo group implies that the drug prevents two outcome events per 100 treated for 1 year, corresponding to an NNT of 50 (1/0.02). That is, 50 patients need to be treated continuously for 1 year to prevent one patient incurring the outcome event. The NNT is now used extensively in randomised trials and observational studies to provide an additional and user-friendly measure of the impact of a drug or treatment on a given disease outcome.10 ,15

While the calculation of the NNT as shown is arithmetically simple, this is only so when trials follow all patients up for the same time and observe a single outcome event per patient. In practice, however, most trials involve unequal follow-up times while some will study recurrent outcomes with multiple events, such as COPD exacerbations. The NNT calculation is then not as straightforward in these situations.12–14

Unequal follow-up and single event: patient-based NNT

When follow-up times vary between patients, the CI of an outcome event cannot be calculated as a proportion of subjects. It must instead be based on the Kaplan–Meier approach, which estimates the CI accounting for variable follow-up.16 The NNT can then be directly computed by inverting the difference in the CI of the outcome between the two groups at the desired time point of follow-up from the Kaplan–Meier curves, or alternatively, approximated from the HR.17 It will estimate the number of patients that need to be treated by the drug to prevent one patient with the outcome over the given follow-up time. An example is the Kaplan–Meier curve provided for time to pneumonia in the INSPIRE trial.9 From the curves, the 2-year CI of pneumonia is 0.094 and 0.049 for fluticasone–salmeterol combination and tiotropium respectively for NNT=1/(0.094–0.049)=22 patients for 2 years to induce a pneumonia. The curves also permit the NNT to be computed at any other time point during follow-up, for example at 1 year, NNT=1/(0.06–0.03)=34 patients for 1 year to induce a pneumonia.

Unequal follow-up and multiple events: event-based NNT

With outcomes involving an event that can recur multiple times during follow-up, such as COPD exacerbations, most trials use the incidence rate (IR) to quantify the frequency. It is computed as the total number of events divided by the total amount of person-time, to account for varying follow-up times. The effect of the drug treatment is then measured by the difference in IRs between two groups, namely IR₀−IR₁, where 0 represents the reference treatment or placebo and 1 the treatment under study. This difference represents the IR of prevented events per person-moment. Several studies have, in this case, used an alternative formula to compute the corresponding NNT as 1/(IR₀−IR₁). This NNT, based not on the CI but on the IR, was interpreted as the NNT to prevent one ‘event’ over a given time period.11

An example is the trial by Anzueto that randomised close to 800 patients with COPD to either the fluticasone–salmeterol combination or salmeterol alone, and followed them for 52 weeks.6 With varying follow-up times, the IR of moderate or severe exacerbations, which could occur more than once during a patient's follow-up, was 1.59 per patient-year in the salmeterol group and 1.10 in the fluticasone–salmeterol group. The authors thus computed the NNT as 1/(1.59–1.10)=2.04 rounded to 2, which was interpreted as ‘two subjects need to be treated for 52 weeks … in order to prevent one exacerbation per year’.

This alternative formula, called the event-based NNT, has been criticised since it is not based on patients but on person-time, making the interpretation inappropriate.12 Indeed, in an example devised by Aaron and Fergusson, this approach based on rates can produce illogical results such as 0.5 subjects need to be treated for 1 year to prevent one exacerbation. Clearly, treating a half of a patient appears nonsensical. Of course, the rates can be modified and the NNT interpreted as ‘1 subject needs to be treated for 1 year to prevent 2 exacerbations’, but then we are deviating from the meaning of the NNT, which relates to preventing one patient with the event.

Proper use of the event-based NNT

Because the event-based NNT calculation is based on the IR and not the CI, its use and interpretation must be consistent with the definition of a ‘rate’. It is fundamental to understand that the rate is an ‘instantaneous’ measure of the incidence of an event. Thus, even though the Anzueto trial described above reported IRs of COPD exacerbation of 1.10 and 1.59 per patient-year, its proper usage should be instantaneous. To accomplish this, one should use the smallest measurable time unit in such studies, such as the day, as the instantaneous measure of time. The fact that a patient will not have two exacerbations on the same day makes the day a sufficiently small time unit. Thus, the more proper presentation of the rates of 1.10 and 1.59 per patient-year should be 0.0030 and 0.0044 exacerbations per patient-day respectively. Of course, in studies with much shorter follow-up, such as studies of inpatients, a smaller time unit such as the hour can be used depending on the outcome.

By using the instantaneous version of the rate with the smallest measurable time unit, it is then appropriate to use this alternative formula for the NNT. However, this NNT can only be interpretable over the instant-defining time interval. For example, the TORCH study reported IRs of COPD exacerbation of 1.13 and 0.85 per patient-year in the placebo and fluticasone–salmeterol combination group respectively. The NNT was computed as 1/(1.13–0.85)=3.6 rounded to 4 and inaccurately interpreted as ‘NNT of four to prevent one exacerbation in 1 year’.1 Instead, the corresponding instantaneous rates are 0.0031 and 0.0023 per patient-day corresponding to an NNT of 1250 patients treated with fluticasone–salmeterol on a given day to prevent one exacerbation on that day. Table 1 shows the proper instantaneous event-based NNT for the three trials of fluticasone–salmeterol that computed the NNT using the yearly event-based NNT.

It is important to note that the instantaneous event-based NNT cannot be converted directly to a longer term NNT. Thus, for example, it is incorrect to take the NNT of 1250 patients treated with fluticasone–salmeterol on a given day to prevent one exacerbation from the TORCH trial, divide it by 30 and claim that the resulting value of 42 corresponds to the NNT for 1 continuous month of treatment to prevent one exacerbation. The next section provides formulae that permit the conversion to different time periods.

Converting event-based to patient-based NNT

As noted previously, the fundamental flaw with the event-based NNT as it is commonly used in many studies is that it is not based on the instantaneous rates, but rather on rates that have been extrapolated to a longer time span. It is simply incorrect to convert directly the event-based NNT to a patient-based NNT. However, by first converting the instantaneous rate into a CI function over time, it becomes possible to then compute the corresponding patient-based NNT. To do this, an approximation based on the relation between the Poisson and exponential distributions can be used with the following formula: where CI is the cumulative incidence of the outcome event up to time t and IR is the incidence rate of outcome events measured in the same time units as t.14 Note that the CI and IR are practically equal when the IR is rare over the desired time span (IR less than 0.10 or even 0.15 per patient), making this conversion unnecessary. However, with more common outcomes such as COPD exacerbations (eg, IR=1.13 per patient per year in the TORCH study's placebo group), this formula becomes essential in the absence of the preferred Kaplan–Meier CI. It is also important to note that this formula is only approximate as it depends on the strong assumption that the events follow a Poisson distribution, which is not often the case. In fact, we know that this is clearly not so for COPD exacerbations as they tend to cluster in time and become more frequent.18–20 In this case, the formula will provide somewhat biased estimates of the NNT, which can only be properly computed from the Kaplan–Meier CI estimates.

In the example of the Anzueto trial, the authors used the rates of exacerbations (per patient per year) of 1.10 and 1.59 for the fluticasone–salmeterol and salmeterol groups respectively, leading to a reported ‘NNT’ of two as noted above. Instead, the conversion of the daily instantaneous rates of 0.0030 and 0.0044 per patient into CIs would give corresponding 1-year CIs of 0.67 and 0.80, leading to a NNT of eight patients who need to be treated for 1 year to prevent one exacerbation. In fact, the paper actually provided Kaplan–Meir curves for the time to the first exacerbation, with 1-year CI estimates of 0.60 and 0.67 for the fluticasone–salmeterol and salmeterol groups respectively, corresponding to a NNT of 14 patients needing to be treated for 1 year to prevent one exacerbation. This study illustrates well the inappropriateness of the event-based NNT (2 instead of 14) and to a lesser extent the bias from the deviation from the Poisson assumption needed to convert the rate to the CI, giving an NNT of 8 instead of 14.

COPD exacerbations versus pneumonias

Table 2 displays the properly computed or approximated NNTs for five trials assessing the fluticasone–salmeterol combination against a long-acting bronchodilator. The NNT to prevent one COPD exacerbation and to induce one pneumonia case are presented side by side. While some studies suggest that around twice as many patients need to be treated to induce a pneumonia compared with preventing a COPD exacerbation, the longer term TORCH and INSPIRE trials suggest an opposite benefit–risk impact.