In recent years, clinical decision making has been directed away from the doctor’s clinical experience towards a paradigm based on the evidence from randomised controlled trials (RCTs). The results of large RCTs have been translated into guidelines containing evidence-graded recommendations, which the clinicians are encouraged to accept as the basis of good clinical practice. However, RCTs are only able to guide clinical decision making when trials are well designed, have clinically relevant outcome measures and the subjects in the trial are representative of the range of real-life patients managed by the doctor.¹ This last requirement is not always met, and it is recognised that older adults, women and ethnic minorities may be under-represented in RCTs.^2–,⁵ Design considerations often lead to RCTs that are performed in highly selected patient populations, such as those with the most typical features of a disease or those most likely to respond to the intervention being studied. This may be necessary to initially assess the efficacy of an intervention—for example, in a phase II clinical trial. However, when these same design considerations are applied to phase III and IV clinical trials, they may result in the exclusion of many subjects in whom treatment may be potentially useful,¹ thereby restricting the generalisability of the trial results.

Asthma is a heterogeneous disease with a wide range of clinical phenotypes,⁶ hence, it is not surprising that there is evidence that many individuals with asthma are not eligible for RCTs owing to highly selective inclusion and exclusion criteria.^7,8 This makes it difficult for clinicians to know to what extent the evidence for the safety and effectiveness of an intervention applies to a given individual. The proportion of individuals with asthma who are eligible for the major asthma RCTs from which the clinical evidence is derived is not known. In this population-based survey of respiratory health, we determined the proportion of individuals with asthma who would have met the eligibility criteria for the RCTs that form the basis of asthma consensus guidelines.

METHODS

RESULTS

A total of 2319 participants responded to the postal survey, representing a response rate of 78%. Of these respondents, 749 participants completed the detailed questionnaire and satisfactory pulmonary function testing, and formed the study group (fig 1⇓). Compared with the 1570 survey respondents who were not included in the study group, the study group had a higher rate of asthma diagnosed by the doctor (23.1% v 17.3%), more men (53.7% v 44.3%) and ex-smokers (41.4% v 35.3%). There were no significant differences in the prevalence of chronic bronchitis or emphysema diagnosed by a doctor.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

 Flow diagram of the Wellington Respiratory Survey.

Of the 749 participants in the study group, 179 (24%) met our criteria for current asthma and 127 (17%) met our criteria for current asthma on treatment. Among the 179 participants with current asthma, 67 also met the criteria for chronic obstructive pulmonary disease (COPD), defined as an FEV₁/forced vital capacity ratio <0.7 postbronchodilator.¹² Of these 67 participants, 29 had a tobacco cigarette history of >10 pack-years. Table 1⇓ presents the characteristics of the participants with current asthma.

There were 215 individual references in the relevant chapter of the GINA guidelines, from which 17 qualifying RCTs were identified and included in the analysis (fig 2⇓).^13–,²⁹ Table 2⇓ gives the characteristics of the included RCTs. The number of participants screened was stated in 8 of the 17 RCTs included in the analysis.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

 Flow diagram for the selection of randomised controlled trials (RCTs). GINA, Global Initiative for Asthma.

Inclusion criteria used in all 17 RCTs were a diagnosis of asthma, age greater than a lower age limit and bronchodilator reversibility. Other inclusion criteria were a specified FEV₁ range in 16, specified inhaled corticosteroid use in 12, specified symptoms or use of rescue drugs in 9, age less than an upper age limit in 7, peak flow variability in 4 and other inclusion criteria in 4 RCTs. Exclusion criteria used were recent respiratory tract infection or asthma exacerbation in 13, potentially confounding use of drugs in 11, comorbid conditions in 9, more than a specified amount of smoking in 7, pregnant or lactating female participants in 5 and other exclusion criteria in 6 RCTs.

The proportion of participants with current asthma who met the eligibility criteria for these 17 RCTs ranged from 0% to 36%, with a median of 4% (table 3⇓). The proportion of participants with current asthma on treatment who met the eligibility criteria for these trials ranged from 0% to 43%, with a median of 6% (table 3⇓).

Table 4⇓ shows the proportion of participants with current asthma who did not meet common eligibility criteria. The most selective criterion was bronchodilator reversibility, which excluded either 71% or 76% of participants with current asthma, depending on whether 12% or 15% reversibility in FEV₁ was required. The requirement for peak flow variability of ⩾20% resulted in the exclusion of 66% of participants with current asthma.

DISCUSSION

This study shows that the major asthma RCTs have been performed on highly selected patient populations, with a median of only 4% of participants with current asthma in our community being eligible. Similarly, a median of only 6% of participants with current asthma on treatment would have been eligible for these RCTs. These findings suggest that although the treatment recommendations of major asthma guidelines have a strong scientific evidence base, they may be limited with respect to the external validity of the RCTs from which they are derived.

We made no attempt to perform a complete search of the asthma literature, and probably RCTs have been reported elsewhere that are more inclusive of participants with current asthma than those examined. However, the RCTs selected here are those that provide the evidence for the treatment recommendations of the GINA guidelines,¹¹ and hence have a direct effect on asthma management worldwide. Using actual RCTs rather than a hypothetical “typical’ RCT⁷ has allowed a realistic estimate of the degree of selectivity of existing asthma trials. It also allows comparison of the degree of selectivity between different drugs for asthma or drug indications—for example, there was only one qualifying RCT comparing theophylline with another agent, for which none of the 179 participants with current asthma in our study were eligible.¹⁷

We were not always able to determine from our survey data whether a subject with asthma met a particular criterion, as many trials used criteria for exacerbations, symptom scores and measures of drug use that we were not able to duplicate. Where this occurred, participants were deemed to remain eligible by the criterion we could not assess. Hence, our estimates of the proportion of participants with asthma eligible for a given trial are maximum values, and the true degree of selectivity of these RCTs is probably greater than we have shown.

Our definition of asthma did not exclude those with concomitant COPD, defined as a post-bronchodilator FEV₁/forced vital capacity <0.7.¹² About one third of the participants with asthma met these criteria, most of whom did not have a significant smoking history. As a result, it is probable that this group with concomitant COPD was predominantly made up of participants with asthma, who had developed an irreversible component to their airway obstruction. Importantly, they represented a group of participants that had mostly received a doctor’s diagnosis of asthma and had been prescribed asthma treatment based on trials that largely excluded them.

The most common reason that participants with current asthma were not able to meet RCT eligibility criteria was the need to show bronchodilator reversibility. Application of this criterion resulted in only about a quarter of participants with current asthma in our survey being eligible for the RCTs included in the study. This proportion was greater than that observed from a similar study from Australia, in which only 7–18% of participants with current asthma showed bronchodilator reversibility depending on the criteria used.³⁰ These observations are likely to be due to the widespread use of inhaled corticosteroid treatment, resulting in good asthma control and associated reduction in lung function variability in asthma. The use of bronchodilator reversibility criteria in asthma RCTs may be justified on the basis that it provides the greatest opportunity to determine the maximum efficacy of a therapeutic agent. It also identifies individuals who do not have optimal asthma control, and as a result could be considered suitable for the addition of another therapeutic agent. This has led to novel study designs in which participants reduce or withdraw their inhaled corticosteroid treatment to demonstrate unstable asthma and its associated lung function variability.^31,32

Other eligibility criteria, such as the requirement that participants be non-smokers, also tend to produce a more homogeneous study population. This has the advantage of reducing the likelihood of a participant having concomitant COPD and limiting the number of variables apart from the RCT intervention, again maximising the likelihood of demonstrating a therapeutic effect specific to asthma. However, the criterion that participants be non-smokers resulted in the exclusion of 30% of the participants with current asthma in our study, representing a major group in which drug efficacy would not have been assessed. The importance of this limitation is evident from the experience with inhaled corticosteroids in asthma, where the studies showing a reduced efficacy of inhaled corticosteroids in smokers were not undertaken until >25 years after the introduction of these agents.^33,34

In our study group, the prevalence of wheezing and asthma diagnosed by a doctor was relatively high, with rates of 28.8% and 23.1%, respectively. These findings reflect the high prevalence of asthma in the New Zealand population, consistent with previous surveys that have reported rates of wheezing between 26% and 30%.^35,34,37 Owing to the high survey response rate and the similarity between survey responders and the study group that undertook the investigative procedures, any effect of non-response bias is likely to be small.

In summary, we conclude that the degree to which the results from asthma RCTs apply to individual patients cannot be assessed directly and the clinician cannot assume that their patient will respond to a drug in the same way as trial participants. As a result, clinicians should consider that the treatment recommendations of major asthma guidelines may be limited with respect to the external validity of the RCTs on which they are based. This does not mean that the results of these RCTs are not generalisable to the wider community of individuals with asthma, but rather that the degree of generalisability is uncertain. We encourage the inclusion of a wider range of participants with asthma in future clinical trials.