Article Text
Statistics from Altmetric.com
We read with interest the article by White et al on the use of homeopathy as an adjunct in the treatment of childhood asthma.1 We also obtained negative findings in an open study in which we assessed the effects of homeopathy on spirometry and exhaled nitric oxide (eNO) in children with stable asthma.
Twelve asthmatic children (4 boys, median age 13.5 years, range 7–18) who satisfied the following inclusion criteria were recruited: (1) stable asthma with no clinical indication for change in treatment, on any dose of inhaled corticosteroid and any other asthma medications; (2) raised eNO level at the start of the study despite clinical stability; (3) identifiable sensitivity to house dust mite (HDM, n=3) or cat and HDM (n=9) by history and skin prick test (SPT); (4) no hospital admission or emergency department attendance for asthma in the previous 3 months; (5) no history of consumption of oral corticosteroid in the previous 3 months; (6) no homeopathic treatment within the previous 6 months, allergen desensitisation within the previous year, or HDM avoidance measures or removal of household pet to which the subject had a positive SPT in the previous 3 months.
At baseline all recruited patients underwent SPT if this had not been done within the previous 2 years, eNO measurement (NIOX, Aerocrine, Sweden), and spirometric testing (Vitalograph, Buckingham, UK) measuring forced expiratory volume in 1 second (FEV1). The mean of three best efforts was recorded and the result was expressed as percentage predicted. The homeopathic remedy was prescribed according to the child’s SPT result. This was a preparation of HDM or cat dander (or both, if appropriate) in the form of two lactose globules. The preparation was made up according to the principles laid out in the British Homeopathic Pharmacopoeia. The patients were told to take the globules daily for the next 4 weeks while continuing with the same conventional asthma treatment. A diary was given to each child to encourage compliance and to document any breakthrough symptoms or side effects from the remedy during the study period. The subjects were told to return for eNO measurement and spirometric assessment after 4 weeks (visit 1) on the homeopathic remedy, and to return again 4 weeks later (visit 2) to assess the response after stopping the remedy. The spirometric test results of one patient from the first and second visits were missing.
No side effects were reported and all subjects were compliant with the homeopathic remedy. Using the Wilcoxon signed ranks test, there was no significant difference at baseline and at visits 1 and 2 in FEV1 (86% (interquartile range (IQR) 81.1–93.3) v 89% (85.0–100.0) v 85% (74.0–89.0), respectively) and eNO (54 ppb (IQR 36.2–99.6) v 68 ppb (37.0–87.0) v 76 ppb (43.6–131.4), respectively). This could be because of the small sample size or because the homeopathic remedy genuinely did not have any anti-inflammatory effect.
This study provides important baseline data for the calculation of the sample size needed to carry out a randomised, placebo controlled, double blind study. A sample size of 65 subjects per treatment arm would have 80% power to detect a difference of 10% in mean FEV1, assuming a standard deviation of difference of 28.86, using a paired t test with a two sided significance level of 0.05.
References
Homeopathy deserves to be scientifically appraised by good quality studies and the results published without bias which could distort future meta-analyses. The study on childhood asthma by White et al1 published recently in Thorax has critical flaws which seriously undermine its conclusion. The main weaknesses of the study, which were mentioned by the authors but not given due attention, were the limitations of the primary outcome measure and the mildness of the children’s asthma. However, there is also concealed selection and measurement bias which could have been prevented when planning the trial.
Available guidelines for the diagnosis of asthma were not properly used for inclusion of patients, leaving room for doubt as to whether or not the included patients had asthma. Classification of asthma severity could be established at entry by using published international paediatric asthma consensus or guidelines.2 Better physiological measures could have been used—peak expiratory flow results are less reliable than forced expiratory volume in 1 second, which is the most reproducible pulmonary function parameter.3 All patients were using β adrenergic inhalers and more than two thirds had been prescribed inhaled steroids at baseline and were well controlled; at least 50% of patients in the homeopathic group had had no asthma event in the previous 12 months, suggesting a design bias against homeopathy (ceiling effect). Sample size was calculated without a pilot study and did not allow for the fact that comparisons of the impact of asthma treatments on quality of life are likely to involve relatively small effect sizes even when one treatment is clearly superior.4
My paper on the safety of homeopathy5 is misquoted; it does not in any way imply that the rate of exacerbations is a “hallmark of successful treatment”. Instead, I stated that “one needs to consider the way practitioners are informing patients of the possibility of such aggravations after using homeopathic medicines, thus creating some expectations that will fulfil what was said in the consultation”.
Finally, I cannot agree with the statement that the trial was designed with the input of experienced homeopathic practitioners for optimal conditions: individualised prescription of homeopathic medicines needs a good medical understanding of asthma to discriminate between disease-specific and patient-specific or peculiar symptoms. Treatment was by non-medically trained homeopaths without proper medical supervision, and this has implications on the selection of medicines. Medical doctors prescribing homeopathic medicines know what the patient has in terms of conventional diagnosis and can distinguish features typical of the disease from those specific to the individual patient. This was not adequately considered by the authors in planning the study.
Taken together, these biases seriously undermine the validity of the claimed results. Such shortcomings should be eliminated from future trials of homeopathy for asthma published by respected journals such as Thorax.
The efficacy and clinical effectiveness of homeopathy engenders considerable debate; it is therefore essential that clinical trials are accurately interpreted and reported. The recent publication by White et al1 has highlighted this issue.
The study—which assessed classical homeopathy as an adjunctive treatment for childhood asthma—concluded that, based on the primary outcome (the active quality of living subscale of the Childhood Asthma Questionnaire), classical homeopathy was not superior to placebo. We disagree with this conclusion. The scale used to assess the primary outcome was inappropriate; it does not distinguish between asthmatics and non-asthmatics2 and is more suitable as a cross sectional measure than as a longitudinal outcome, and the ability to identify any therapeutic improvement was severely reduced due to ceiling/flooring effects in both the primary and some secondary outcome data. For example, baseline scores identified that the study population had good quality of life, and that two of the three age groups studied had mild asthma. Any therapeutic improvement would therefore be hard to identify, let alone quantify.
Other design issues were apparent—for example, no data were reported on homeopathic exacerbations (an indicator of the healing response), and the security of blinding was not assessed. Yet, despite these limitations, some encouraging therapeutic effects were apparent. For example, a clinically relevant improvement in asthma severity (unadjusted scores) was seen in two of the three groups, and a favourable pattern in the days off school/days attended was seen in the homeopathic treated children (although no data were presented).
We suggest that a balanced and accurate conclusion to these data would be that no definitive conclusions could be drawn but that further investigation is needed. We therefore hope that the authors’ inaccurate conclusions neither dampen future research nor bias future systematic reviews.3
The paper by White et al1 in the April issue of Thorax purports to show that adjunctive homeopathic treatment has no effect on the quality of life of asthmatic children. The primary outcome measure was the active Quality of Life (QoL) scale of the Childhood Asthma Questionnaire (CAQ). This measure was inappropriate, and no such conclusion is justified. As table 1 shows, the children enrolled had QoL scores virtually identical to those of non-asthmatic children in a large scale validation study cited in the paper by White et al.2 Furthermore, the CAQ QoL scale does not discriminate between non-asthmatic and mildly asthmatic children.
Similar floor and ceiling effects are seen in many of the secondary outcome measures—for example, in the homeopathy group at entry the peak expiratory flow rate was 100.4% of expected and the median number of asthma episodes in the preceding year was zero. These are all “hard” floor/ceiling effects; no improvement at all could have been expected. There is also a strong suggestion of floor/ceiling effects in other outcomes such as days lost from school, but we cannot be certain from the published data. Other secondary outcomes show relative floor/ceiling effects—for instance, the mean final value in the CAQA parental severity score was 5.5 on a scale of 5–19. Since this was an intention to treat analysis, 20% of the values were simply pretreatment values carried forward.
Other CAQ subscales analysed as secondary outcomes consistently favour homeopathy. For the severity subscales the improvement was statistically highly significant (p=0.01) with 95% confidence intervals not including zero. This again was an intention to treat analysis, and while there are good reasons for performing such analyses, effect size estimates should be based on data for subjects who have actually taken the treatment and had its impact evaluated; in this case, 20% did not. In addition, there was a floor effect (see above) in one of the severity scales. A similar pattern is seen for other subscales of the CAQ, but no statistical analysis is presented.
The question most frequently posed about homeopathy is “is it all placebo effect?” Most meta-analyses have concluded that it is not. If the outcome measures which could not have improved are excluded, the results of this trial accord with those of the largest meta-analysis of homeopathy; they “are not compatible with the hypothesis that the clinical effects of homeopathy are completely due to placebo”.3 The treatment effect size was relatively small, but classical homeopathy is a complex and non-standardised intervention. The practitioners involved had no particular experience of asthma. There is thus considerable scope for refinement.
Regrettably, the conclusions do not adequately reflect the shortcomings of the trial. The authors state that “there was no evidence of a clinically relevant change in quality of life score”, but omit to mention that none was expected since the QoL scores were normal at entry. There is no reference to the many floor/ceiling effects.
Our greatest concern is that the bias in the interpretation of the results will carry through to future meta-analyses and reviews. Methods such as that developed by Jadad et al4 would assess this as a high quality study, and the primary outcome appears to be negative. As we have shown, this interpretation is fundamentally flawed. We believe a correction should be published which should focus on (1) the inappropriate scope of the original conclusions, and (2) clarification of the secondary outcomes and the conclusions drawn from them.
The study by White et al1 of quality of life in children with asthma treated with homeopathy is fatally flawed. The Childhood Asthma Quality of Life instrument used was validated in a study by French et al.2 The children entered into the study by White et al had scores consistent with those of normal children who don’t have asthma. For a statistically significant improvement to occur in this score, the treated group would have to develop scores of around 100%—that is, better than normal non-asthmatic children. This is clearly highly unlikely. In addition, a similar “ceiling effect” applies to the peak expiratory flow readings which at entry were 100.4% and 96.9% of expected for the verum and placebo groups, respectively.
This is a very poor quality trial which does absolutely nothing to further our understanding of the potential value of homeopathic treatment in children with asthma. In fact, the press release from the journal has been picked up by the media and used to support the headline “Homeopathy of no use in asthma”.
Publishing this quality of research at best does not improve our necessary evidence base and, at worst, contributes to the denial of services which may indeed be of value to patients. A close analysis of the study shows that the treatment group had a trend to better outcomes than the placebo group. If this were a pilot study, it would be indicating that there is indeed a potential benefit to asthmatic children from homeopathy which should be investigated with a proper trial of good methodological quality.
Author’s reply
These authors are to be thanked for their contribution to the debate about the interpretation of the results of our trial, and it is conceded that ceiling effects may exist which limit the interpretation of our results. If the children were already effectively medicated, it may not have been possible to show any benefit from homeopathic treatment in quality of life. It would require a much larger study to show any differential change in conventional medication or global indicators, which were absent from the results. Because of ethical issues, it may be difficult to conduct a definitive trial in children with severe asthma.
Leckridge, Fisher et al, and Brien and Lewith suggest that we should have concentrated instead on the small changes in the severity subscales, the estimate of which is not clinically relevant. However, the severity scale measures symptoms only, and active quality of life is much more appropriate for the holistic approach of homeopathy.
The claim by Fisher et al and Dantas that the homeopaths were inadequate to the task is speculative and one we reject. Dantas would prefer us to have used more rigorous criteria for inclusion and assessment, but the study was especially designed to reflect “real life” pragmatically by rigorously applying the criteria used by the GPs, the children, and their families. We cited his paper as the only systematic and objective report on homeopathic aggravations that we were aware of, and if we gave the impression that he stated that aggravations are a hallmark of success, then we regret it.