The Society can congratulate itself on many things as it celebrates its 25th anniversary, but unquestionably one of the ways in which it has made an impact is as a producer of clinical guidelines. Guidelines are the most frequently visited section of our own website, and the BTS/SIGN asthma guideline1 has had more hits than any other on the SIGN site.

Some clinicians do not like guidelines, feeling that they are somehow restrictive or that they promote medical laziness. Some feel that they do not need them. What most people would probably agree is that, if we are going to have guidelines, they should be good ones. The criteria for best practice in guideline production have changed considerably since the Society’s earliest work, and at our current milestone it is appropriate to look forward and consider how our high standards should be maintained. There is insufficient space here for a technically complete presentation on guideline methodology (and I doubt that most readers would want that), so I will simply highlight a few key issues.

TOPIC SELECTION

Keeping guidelines up to date while maintaining high quality is a major task, and any organisation like the BTS with an interest in this area needs to decide how to prioritise and to work on topics which will provide maximum benefit to patients. Guidelines can serve useful secondary purposes, such as demonstrating gaps in clinical evidence and encouraging research to address these, but primarily they should address a defined clinical need.

The main criteria which might be used to select topics are:

• Is the disease common? Small improvements in a common disease area may lead to large overall health gain.

• Is there variation in current practice? This would imply disagreement about best practice, or possibly a degree of ignorance, either of which might be helped by a guideline.

• Is there a substantial, or emerging, body of good clinical evidence on which to base guideline recommendations?

It is a pity, given the effort involved in developing guidelines, that different organisations duplicate each others’ work. Greater collaboration is needed, and indeed seems inevitable as individual organisations reach their maximal capacity for developing new and updating old guidance. The BTS has worked well with SIGN, and has piloted cooperative work with colleagues in Australasia in its guidelines on asthma and diffuse parenchymal lung disease.

COMPOSITION OF THE GUIDELINE DEVELOPMENT GROUP

The most important feature of a guideline group is that it includes those with a strong clinical background in the relevant condition. The idea that only doctors have the relevant expertise has long been disregarded, and it is also generally recognised that there should be patient and carer representation. Patients will need support in issues such as the interpretation of clinical trial data and a group Chair who encourages their full participation. Organisations such as the National Institute for Health and Clinical Excellence (NICE) and the Royal College of Physicians (RCP) have accumulated useful expertise in supporting expert patients.

GETTING THE EVIDENCE

A guideline should be based on the best available evidence. Some would argue that the relevant clinical experts will know the literature well enough to identify the important papers and question the added value (and cost) of a time-consuming electronic search. However, a truly valid guideline would be the same whichever experts produced it, and a standardised search removes one source of variation. Moreover, I have yet to be involved in a guideline where a detailed electronic search did not turn up one or two pieces of evidence which surprised the expert group.

As a minimum, an electronic search should cover the databases MEDLINE, EMBASE and CINAHL, databases of systematic reviews (the Cochrane Library, DARE) and other specialist sources for particular questions—for example, AMED (for complementary medicine) or Psych-INFO. It is possible to look at what other guidelines have said on a similar topic by looking at the National Guideline Clearing House website, and to search for conference proceedings and “grey literature”, although the law of diminishing returns may well start to apply.

Trawling these extensive databases for relevant titles could produce an unmanageable number of papers, and it is important to focus the searches as tightly as possible to the key clinical questions. These need to be defined by the Development Group using their expertise to identify the pivotal decision points in patient management. Once these basic questions are agreed, it is more useful still to put them into the PICO format (population; intervention; comparison; outcome). Thus, a rather vague question such as “Is aminophylline beneficial in acute exacerbations of COPD?” is satisfactory only in terms of the first PICO point (ie, the population is patients with acute exacerbations of COPD). The intervention referred to could be made more specific (ie, intravenous aminophylline). The appropriate comparators and relevant outcomes would be matters of debate by the expert group who may, for example, decide that the only outcomes of interest are mortality and length of hospital stay, or alternatively that changes in blood gases or forced expiratory volume in 1 s are acceptable proxy efficacy measures. This exercise is useful in limiting the number of abstracts and papers identified, but also in concentrating minds on the most important parameters.

ASSESSING THE EVIDENCE

Once papers have been appraised, all those relevant to a particular question must be considered together and conclusions drawn based on the highest quality evidence. For questions of treatment it is accepted that randomised controlled trials (RCTs) should provide the best data, and if there are lots of these and if they are consistent, deriving recommendations should be easy. Even in this situation, however, things may not be straightforward. RCTs are often not the best source of evidence for adverse effects where long-term surveillance data may be more informative. RCTs may disagree and the group must then weigh up the reasons for this, bearing in mind the quality and perhaps the size of the discordant studies. Judgement is also needed if the available RCTs are few or of mediocre quality but there are good big cohort studies addressing the same question, in which case these may over-ride the RCTs. In my experience, a group will usually achieve consensus fairly easily for most questions, but there needs to be an agreed method of resolving conflict. This may be a voting system or a formal consensus technique (eg, the Delphi technique).

Over recent years most people have become familiar with the concept of grading recommendations, and BTS guidelines have done this for the past 10 years or so. Grading is based on the strength of the underlying evidence, so that Grade A recommendations can only be derived from the highest level of evidence in the accepted hierarchy—either RCTs or a meta-analysis of RCTs. There has been unease about this system for some time. It does not accommodate the fact that some clinically important questions, for example on diagnosis, cannot be addressed by RCTs. Furthermore, treatment options which have been around for years and which no one could reasonably question, such as giving oxygen to hypoxic patients with pneumonia, may well not be supported by the rigorous RCTs demanded of any new treatment. The grading system therefore reflects the scientific background but not necessarily the clinical importance of a recommendation. NICE has also become concerned because some end users, faced with having to implement numerous guideline recommendations, have been concentrating only on those graded A, and NICE are about to remove the grading of recommendations from their formal guideline documents. There is a down side to this, in that the grading clearly demonstrates where there are gaps in the evidence, an important secondary function of a guideline.

A new system of grading recommendations is under development named, Ronseal-style, GRADE.2 This attempts to reflect both the strength of evidence and the degree of clinical certainty behind a recommendation, and may well feature in future guidelines.

COST EFFECTIVENESS

A purist might argue that economics has no place in a clinical guideline. However, if a guideline is going to be applicable in practice—and there is no point developing one that isn’t—then some consideration of resources is inevitable.

At a simple level, a guideline might suggest the use of the cheapest antibiotic to which an organism (or likely pathogen) is sensitive, or the cheapest inhaled steroid device which a patient can use successfully. But what if the relapse rate with cheap antibiotic A is twice that of more costly antibiotic B, requiring a second course of antibiotics twice as frequently? And what if C difficile is more common with antibiotic A, with a measurable incidence of prolonged hospital stay as a consequence? The total cost of using the superficially cheaper product may actually surpass that of the more expensive agent.

It is possible to construct complex models setting out all the possible ramifications of a clinical decision, and to attach a cost to these options. It is, however, time-consuming to build up models from scratch for every possible recommendation, and it is acceptable to focus only on particular issues. For example, there is little point in performing complex analysis where one option clearly gives better clinical outcomes than the alternatives, unless the superior option is particularly expensive. The definition of “expensive” is, of course, debatable, and one reason many feel uncomfortable with health economics is that it forces decisions about the amount one should be prepared to pay for clinical benefit. Quite properly, the decision about such thresholds is not the province of the guideline group themselves.

Detailed health economic input is relatively new to many guidelines, and there are issues around the integration of this with the purely clinical work which are not fully resolved.3 However, it will be increasingly difficult to produce acceptable guidelines without some consideration of economic consequences.

DISSEMINATION AND IMPLEMENTATION

Although there is a substantial literature about implementation of guidelines,4 it is still not clear how best to ensure that this happens. Producing clearly written material, in different versions for different audiences, is important but not sufficient by itself. Various methods have been tried to enhance take-up of guidance using both carrot and stick, but there does not seem to be a generalisable consistent means of achieving success. Overall, the best results have been obtained where it has been possible to empower patients to promote guideline adherence, but this is difficult to organise except in very simple situations such as encouraging influenza vaccination.

The BTS has shown that dissemination based around genuinely helpful materials can work well. The COPD Consortium, founded to promulgate the 1997 COPD guideline, was hugely successful. The dissemination efforts which followed the BTS/SIGN asthma guideline launch in 20035 stimulated over 200 000 downloads in the first 18 weeks, and larger numbers still if the associated teaching materials are taken into account.

CONCLUSION

It is harder to produce methodologically acceptable guidelines than it was 10–15 years ago, but important that technical validity is not achieved at the expense of clinical usefulness. Although there are now national and international organisations involved in guideline production, I believe it is important that specialist societies continue to be involved. They can best judge where guidance is necessary on the “smaller” topics; for guidance on the big diseases, where those outside the specialty have a significant stake and where the amount of evidence may stretch resources, the future may lie in collaboration with others, but our input is still crucial. The BTS can look back with pride on its previous work; it is also well placed to move forward as guideline production evolves.