Continuous positive airway pressure (CPAP) therapy has been regarded for more than a decade by those in the field as the treatment of choice for the sleep apnoea/hypopnoea syndrome (SAHS). Nevertheless, two recent systematic reviews1 ,2 have appropriately pointed out that there is a shortage of robust evidence that CPAP benefits medium and long term outcomes in SAHS. However, the striking aspect of the two reviews is their diametrically opposed conclusions about the efficacy of CPAP, one concluding from the evidence that CPAP was indicated in patients with more than 20 apnoeas + hypopnoeas per hour of sleep plus daytime sleepiness,1 whereas the other concluded that “the studies do not provide sufficiently robust evidence for the effectiveness of CPAP”.2 There are differences in the design of these reviews which may account for this disparity.

Systematic review should be impartial. The review by Wright and colleagues3 was funded, at least in the initial stages, by purchasing authorities concerned about the increasing costs of provision of sleep services and CPAP. Sackett has commented that there is a “fear that evidence based medicine may be hijacked by purchasers and managers to cut the costs of healthcare”.4 The study by Wright et al 2 would appear to be one such example. In contrast, the Australasian study1 was commissioned by the Australian Medical Research Council and the New Zealand Ministry of Health.

Evidence based medicine has been defined as “integrating individual clinical expertise with the best available external clinical evidence from systematic research”, where clinical expertise is “the proficiency and judgement that individual clinicians acquire through clinical experience and clinical practice”.4 Wright and colleagues2 did not include in their panel any physicians with any experience of sleep apnoea; indeed, only one of the panel of five was medically qualified, a physician in public health medicine. In contrast, the Australasian review1 had two physicians with experience in sleep medicine. Appropriately, these two were in a minority, the remainder of the panel comprising two physicians with broad experience of evidence based medicine and one anaesthetist. Furthermore, the recommendations of this panel were then revised by a panel of 10, none of whom were sleep physicians, to further guard against advocacy by the sleep clinicians.

Wright et al 2 ignored the many acute studies showing that CPAP was effective in preventing obstructive apnoeas and improving sleep quality, whereas the Australasian study1included such investigations. The authors of the latter study also carried out a meta-analysis of medium term studies performed without placebo control and concluded that these provided evidence of a substantial impact of CPAP which was too great to be solely a result of bias.

The authors of both reviews agreed that at the time there was only one placebo controlled study examining the efficacy of CPAP,5but came to radically different conclusions about its value. The Australasians1 found “no major methodological threats to its validity” while Wright et al 2 concluded that the “study had important weaknesses”. Although these are not clearly specified, there appear to be three areas of concern: the placebo used, the lack of washout period, and the test used for differential carryover.

Engleman and colleagues5 used an oral placebo which was actively advocated to patients as an agent which might improve upper airway muscle function, this being done with approval of the local ethics advisory committee. This type of placebo was chosen after careful consideration of the alternative approaches, including the possible use of a CPAP mask attached to a CPAP machine delivering the lowest possible pressure, so-called “sham CPAP”. Sham CPAP was rejected by Engleman et al for five reasons: (1) the wearing of a mask at night is likely to interfere with sleep, thus impairing daytime function and so falsely biasing in favour of active therapy; (2) CPAP at a subtherapeutic level had been reported at the time the study was designed to cause dangerous hypoxaemia6; (3) sham CPAP would be readily identified by patients, thus unblinding a crossover study; (4) sham CPAP at a pressure sufficient to wash out carbon dioxide may prevent upper airway narrowing and thus be an effective therapy in some patients; and (5) it was not clear whether patients would comply with an obtrusive placebo which could be detrimental to sleep and well being.

Wright et al 2 also criticised the absence of a washout period between the two limbs in Engleman’s study, but this could not be regarded as an “important weakness” of the study for two reasons. Firstly, any carryover effect of the active treatment would have biased against the positive findings of benefit with CPAP and, secondly, Engleman et al did not measure any of the outcomes until 28 days after crossover, whereas the effects of CPAP wear off within a day.7

Wright’s criticism of the absence of tests of differential carryover are also not “important weaknesses” as any carryover would again have biased against the positive findings. Furthermore, carryover was sought by analysis of variance for all of the outcome measures and only found for the one variable reported, necessitating a different statistical approach for this variable.5

Thus, it seems the “important weaknesses” probably amount to concerns about the use of an oral placebo. The Australasian study expressed concern about the use of an oral placebo as well, but did not consider this invalidated the conclusions of the study. We acknowledge there is no perfect placebo for CPAP but feel that an oral placebo, as used in our study, is one of the better—or perhaps the best—options available and gives valid results.

Our major concern about the evidence of efficacy of CPAP is the dearth of corroborative data, a concern shared by the authors of both reviews.1 ,2 Since these reviews were conducted, one further placebo controlled study has been published by Engleman and colleagues8 which indicates improvements in symptoms, mood, and mental flexibility with CPAP, even in patients with only mild SAHS (5–15 apnoeas + hypopnoeas/hour). Having carried out searches using Medline and Embase, we were unable to identify any other controlled studies of CPAP in the literature. By contacting those involved in therapeutic research in SAHS we identified four further studies yet to be published. (1) The further report by Englemanet al 9 of 23 patients with AHI of >15 who showed significant benefits from CPAP. When combined with the 25 patients in their original report who underwent an identical protocol,5 data on 48 patients with “severe” SAHS (AHI >15) show that CPAP improves symptoms, mood, subjective and objective sleepiness, and a range of cognitive functions (table 1). To place these findings in an appropriate context, the magnitude of the changes observed was as large as those reported for stimulant therapy in narcolepsy.11 It must be stressed that this analysis also strongly illustrates the learning and placebo effects found in patients treated with CPAP, supporting the need for randomised controlled trials expressed in both previous reviews.1 ,2 (2) In a second study of mild SAHS (defined as AHI of 5–15 plus subjective sleepiness), using a different range of tests but a similar oral placebo controlled design, Engleman et al 12again found improvements in symptoms, subjective sleepiness, quality of life, and cognitive function. (3) Badia et al 13 found significantly greater improvements in symptoms and quality of life with CPAP in comparison to conservative therapy which included a weight loss programme in a parallel group study. (4) Stradling et al 14 reported significantly greater improvements in both subjective and objective sleepiness with active CPAP compared with sham CPAP in a parallel group study.

Thus, there are now five controlled studies all showing consistent improvements in symptoms and daytime function with CPAP in patients with SAHS. We were unable to identify any adequately powered and controlled studies yielding negative results. This consistency across five studies from three centres using three different control methodologies is strong evidence for efficacy of this therapy and is in keeping with the many uncontrolled reports received both from patients15 and physicians in the field. There is therefore a need for the benefits of CPAP therapy to be acknowledged by purchasers of health care so that sufferers from the syndrome can receive the benefits of optimal care.