An important article by Peter Rothwell¹ published in the Lancet in 2005 eloquently summarised issues concerning the external validity of randomised controlled trials (RCTs). The issue underlying the question “to whom do the results of this trial apply?” is sometimes referred to as generalisability and reporting on this is now a standard requirement for articles published by many journals directed at a general audience.

Rothwell summarises several factors that must be taken into account when considering generalisability, including the trial setting, patient selection, the characteristics of the patients randomised, the type of outcome measures employed, the nature and duration of follow-up, and the completeness of adverse event reporting. Further, differences between the trial protocol and routine local practice must be taken into account, especially the possibility, given the likely time elapsed between the trial design and reporting, that considerable therapeutic or diagnostic advances have occurred in the interim.¹

A further important development in respect of clinical practice is the emergence of widely disseminated guidelines for the management of common medical conditions. Something of an industry has developed in this regard, with national and international agencies taking a leadership role.² Nowadays, published guidelines are strictly evidenced-based and such evidence is graded for quality on the basis of criteria such as trial design, method of analysis and relevance of outcomes. Champion among recommendations are those based on large, top quality RCTs.

With regard to asthma management, the guideline effort has been prominent and strongly supported by relevant professional societies. International experts have gathered together under the Global Initiative for Asthma brand to develop, promulgate and assist with the implementation of asthma management guidelines worldwide.³

In the case of these asthma management guidelines, the question arises—are they based on relevant, well designed RCTs that can be reliably applied to individual patients presenting for asthma care? This question is addressed in this issue of Thorax in a study performed by Travers et al⁴ (see page 219). The authors carefully examined the 2005 Global Initiative for Asthma guidelines with respect to drug recommendations for long-term asthma management in adults.³ The individual studies selected were among the larger and most often cited RCTs comparing various approaches to drug management, such as the role of long-acting β agonists, the use of higher doses of inhaled corticosteroids, the benefits of leucotriene receptor antagonists and combined products. Taking the 17 high-quality RCTs underpinning the guideline statements, they examined each study for external validity. This was achieved by assessing the entry criteria for each study and matching these with the characteristics of a randomly selected group of adults with asthma (n = 179) identified in a community survey of the prevalence of asthma.⁵

They found, perhaps not surprisingly, that only a small percentage of their community sample would have met the entry criteria for the RCTs concerned. The percentage eligible ranged from 0% to 36% but in most cases the figure was ⩽10%. Notable among the reasons for ineligibility were lack of bronchodilator reversibility, baseline forced expiratory volume in 1 s too low or too high, lack of symptoms and/or bronchodilator use, not currently requiring inhaled corticosteroids and a history of >10 years tobacco use.

Two findings from the study deserve special mention. First, the study determined that the main reason that subjects with current asthma were not potentially eligible was a failure to demonstrate bronchodilator reversibility (three quarters of those surveyed). It noted correctly that one approach to this, adopted in more recent trial designs, is to withdraw corticosteroid treatment under supervision during the screening or run-in period. In addition to unmasking symptoms and bronchodilator reversibility, this also has the advantage of identifying subjects who do not in fact have current asthma.

Second, it observed that some trial designs failed to identify subjects with concomitant mild to moderate chronic obstructive pulmonary disease or a degree of irreversible airflow limitation. This has obvious implications in terms of outcome measures for the study concerned.

These findings will come as no surprise to any clinician who has actively participated in clinical trials. It is a commonly accepted truism that 10 potential subjects will have to be screened and/or entered into the run-in phase to effect one randomised subject.

What then can be said about asthma management guidelines and the advice therein? First, the authors recommend that a “wide range of subjects be included in future clinical trials”. Although a laudable goal, it seems destined to fail, given the role of national regulatory bodies such as the Food and Drug Administration in mandating clinical trial design. Elderly people, those taking multiple other drugs, subjects dependent on alcohol or with chronic mental disorders and, in many cases, women of child-bearing age are deemed ineligible for RCTs in asthma, especially those examining the role of novel compounds.

Second, the authors imply that prescribers should be aware of the limitations of published guidelines in selecting drug treatment for individual patients with asthma. That is a given fact, and we underestimate the role of the doctor’s clinical judgement with respect to these common prescribing decisions.⁶ The choice of pharmacological agent is just one component of prescribing given the range of presentations available. Fortunately, all the drugs we use most commonly, including inhaled corticosteroids, long-acting β agonists and leucotriene receptors antagonists, have a safety profile that allows the prescriber the comfort of adopting an n = 1 trial approach with regard to long-term management choices. If the chosen agent lacks efficacy or results in intolerable side effects, then another drug can be substituted in most cases. These principles are already embodied in the published guidelines and they are just that—guidelines to be used and interpreted by sensible health practitioners.

As Rothwell suggests,¹ common sense dictates that we should not “throw the baby out with the bathwater”. Guidelines and the RCTs they are based on are an accepted dimension of modern medical practice. However, researchers, funding agencies, ethics committees, medical journals, regulators and, above all, the pharmaceutical companies themselves should take a critical look at clinical trial design to maximise relevance and generalisability.