STUDIES

A total of 251 articles was identified in Medline of which 18 met our inclusion criteria.23-40 Embase provided one additional trial,16 and two additional trials were provided by the database of the pharmaceutical company.41 ,42 The Cochrane Controlled Trial Register did not supply further trials but two more trials were found by searching the references of relevant articles.43 ,44 One additional trial45 was revealed to us by an expert in the field who also pointed out the trial by Shioda40 which we had overlooked in the Medline search. Excluded studies were either not double blind, not randomised, not controlled, did not concern the appropriate age group, or investigated the effectiveness of DSCG in exercise induced asthma; one double blind crossover study was excluded because the results were only partially presented.46 Thus, 24 randomised controlled trials of DSCG as a prophylactic agent in children with asthma were reviewed (table 1). Most of the studies were European (n=14, 10 of which were British) or North American (n=7). Two studies were performed in Israel and one in Japan. Studies differed in design, severity of asthma, number of children included, age of children, administration of medication, and follow up period. In total, more than 1000 children were studied in the 24 trials with sample sizes ranging from nine to 218; about half of the children (11 trials) were of preschool age and half were five years or older (13 trials). Before 1977 the studies included only children aged four years and over. The median duration of intervention (period of active medication in case of crossover studies) was 4 weeks (range 3–26). All studies included children with moderate to severe asthma, and all but one study were hospital based. In nine papers it was unclear whether the population was hospitalised or ambulatory, nor was it clear whether and what concurrent medication was permitted during the trial.24 ,26 ,31 ,34 ,36 ,38 ,39 ,41 ,45 Only one trial selected children with moderate asthma through general practitioners.16 Compliance was discussed in only five papers.16 ,23 ,27 ,28 ,42

SIDE EFFECTS

Fifteen of the trials reported side effects, all of which were minor with a low incidence. Cough was most often reported, followed by bitter taste, wheezing, sneezing, throat irritation, and perioral eczema. Some studies did not specify the kind of adverse effects but merely stated “minor”.

OUTCOME

A positive outcome was concluded by the authors in 16 of the studies and in three the outcome was partially positive, depending on age26 and outcome measure.39 ,45 Five of the trials had an equal outcome. No studies in favour of placebo treatment were found.

METHODOLOGICAL ASSESSMENT

The results of the methodological assessments are given in the last column of table 1.The Chalmers scores ranged from 24% to 79% with a mean score of 43% indicating mediocre methodology. The most prevalent methodological shortcomings were in the areas of compliance, selection and inclusion, and statistics and analysis. The descriptions of blinding and intervention were relatively good. Fourteen studies were published before 1980 and the mean method score for these was similar to that for the 10 trials reported after 1980. Our own recent study scored highest; to control for reviewers' review bias the independent observer re-evaluated our study which resulted in similar scores. The Jadad scores ranged from 2 (five studies) to the maximum of 5 (one study).

STATISTICAL POOLING

For most studies the data were insufficient to allow a formal meta-analysis with odds ratios. Two studies did not give symptom score data or data that could be converted into symptom scores.32 ,33 Figure 1 presents two graphs of 95% CIs of differences in mean symptom score (placebo minus DSCG) for each study. The CIs are ordered according to year of publication of the study. In eight studies the CIs were calculated using the separate symptom score of wheeze and cough, and in 12 studies we used daytime symptoms. In two studies36 ,39 only the results for cough were presented; data on wheeze were not published because the difference in treatment effect between placebo and DSCG was not significant.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

95% confidence intervals of absolute difference for (A) cough and (B) wheeze in DSCG group compared with placebo.

The χ² test rejected the hypothesis of homogeneity of the study results (p<0.001), both for the absolute and the relative outcome measures. Taking heterogeneity into account, the pooled 95% CIs for wheeze and cough were 0.11 to 0.26 and 0.13 to 0.27, respectively, and the pooled 95% CIs for RIM were 0.16 to 0.37 and 0.19 to 0.36. The heterogeneity was not related to the placebo symptom level (that is, severity of asthma), method of administration of the medication, duration of follow up, frequency of dosage, methodological scores, and other study characteristics. Only year of publication, age of the children, study design, and duration of follow up were significant predictors of outcome in the univariate regression analysis. The multivariate regression analysis showed that only year of publication was a significant predictor of effect size: older studies were more likely to produce a positive effect of DSCG treatment. Age of the children, which was strongly correlated with year of publication (ρ=0.82), was not significant in this analysis. The pooled 95% CIs under the assumption of heterogeneity and the corresponding tolerance intervals are shown in figs 1 and 2. The latter include zero, both for the absolute treatment effect (–0.11 to 0.48 for wheeze, –0.04 to 0.43 for cough) and for the relative treatment effect (–0.12 to 0.64 for wheeze, –0.02 to 0.57 for cough). The funnel plots are shown in fig 3; the hypothesis of symmetry was rejected for both cough (p=0.095) and wheeze (p=0.01). The asymmetrical funnel plots show that studies with low precision and negative outcome are underrepresented, indicating publication bias.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

95% confidence intervals of % improvement for (A) cough and (B) wheeze in DSCG group compared with placebo.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Funnel plots for (A) cough (22 studies) and (B) wheeze (20 studies).

STUDY CHARACTERISTICS

Although DSCG was indicated as treatment for mild to moderate asthma, nearly all trials comprised hospital based populations of children with moderate to severe asthma. Nine studies administered the medication with a nebuliser. Nowadays, spacer devices are available for young children which are less time consuming and less tedious to use than nebulisers, particularly at home. Metered dose inhalers with spacer devices were used in only two studies.16 ,43Spinhalers were used in 11 trials with older children. The method of administration, a critical factor in delivery of drugs to the lungs, was not a predictor of outcome.

The diagnosis and measurement of asthma in young children is difficult47 ,48 and age effects might reduce or mask the effects of DSCG. However, this is unlikely since, although children's age was a significant predictor for treatment effect in the univariate analysis, the multivariate analysis showed that publication date of the trial was a confounder for age of the children. Duration of follow up in most of the trials did not exceed four weeks which may be too short to assess the effect of the treatment. Although the duration of follow up was not a predictor of outcome, it was notable that none of these short term trials had an “equal” conclusion (12 were positive and two were positive/equal). In seven studies the authors tried to find characteristics or criteria to predict which children would respond to DSCG but none were found.16 ,24 ,25 ,34 ,35 ,37 ,44Silverman et al 35 reported that only the acute protective effect of DSCG in exercise tests predicted the probable success or failure of long term treatment with the drug.

EARLIER REVIEWS

The effects of treatment with DSCG have been reviewed previously. Edwards49 examined the evidence for the anti-inflammatory action of DSCG in adults and children in a large number of controlled and uncontrolled studies but it is unclear how these were selected. Hoag and McFadden50 summarised studies on the effect of DSCG on bronchial hyperreactivity in adults and children. The review by Schweitzer and Brossier Ballano51 discussed three controlled studies assessing the efficacy of DSCG in children aged two years and younger. Finally, Holgate52 reviewed recent trials with metered dose inhalers in children and adults and discussed challenge studies, therapeutic studies, and the long term effects of DSCG. None of these earlier reviews were systematic, assessed the methodological quality, or tried to quantify treatment effects.

METHODOLOGICAL ASSESSMENT

Meta-analyses have been criticised but they are becoming increasingly popular because they give insight into the combined results of trials and provide data for rational decision making.53 ,54 A review of scales and checklists for assessing the quality of randomised controlled trials shows limitations in virtually every scale.55 We decided to use the items proposed by Chalmers15 because these address both the methodology and the presentation of the study extensively. However, we found shortcomings while using the scale, including the lack of attention to sample size, protocol violations, and permitted concurrent medication. The methodological quality of the trials was mediocre, as shown by the mean score of 43% of the maximum attainable. We have to take into account the fact that 13 studies were published before 1980. Informed consent, clear description of inclusion criteria, rejection logs and baseline characteristics, adequate sample size calculation, calculation of confidence intervals, and regression analysis were not common practice at that time.

We also scored the trials according to the criteria of Jadad as this is the only validated scoring system available.15 This scoring system was not found to be very discriminatory because, when dealing with randomised, placebo controlled trials only, as in our review, the outcome score was limited to 2–5. However, half of the studies scored 3 points or less, which is compatible with serious or extensive flaws.

STATISTICAL POOLING

The estimates of the differences between placebo and DSCG treatment for cough and wheeze symptom scores (assuming heterogeneity) were 0.17 and 0.11, respectively, which suggests a small therapeutic effect for DSCG. We doubt whether an effect of this size is of clinical relevance. On the other hand, the overall relative improvement estimates were 27% and 26%, respectively. This considerable relative improvement combined with the minor absolute improvement shows that, overall, the severity of symptoms under placebo treatment was low. Indeed, although most studies in our review included children with severe asthma, the mean daily symptom score in the placebo groups was low (0.8), probably because of dilution by symptom free days which is a common finding in trials of childhood asthma.56 ,57

Care must be taken in drawing definitive conclusions about the role of DSCG in the treatment of asthma in children on the basis of this review for several reasons. Medical literature can be misleading as a result of selective submission and publication of randomised controlled trials showing a statistically significant treatment effect.58The year of publication of the study proved to be the most significant predictor of treatment effect and the asymmetrical shape of the funnel plots suggests bias; studies with a positive treatment effect had relatively little precision and most were performed before 1980. Trials with little precision and a negative outcome have probably not been submitted or published, which is understandable in a period when a newly developed drug is tested and marketed. Our meta-analysis is therefore optimistic in its outcome; the addition of unpublished negative trials would change the results towards an even more negative conclusion.

The trials were heterogeneous in their treatment effects. Refraining from pooling was seriously considered, given the heterogeneity and the suspected publication bias. Nevertheless, we calculated the pooled confidence intervals and the tolerance intervals because these are useful summary measures. In cases of heterogeneity, in particular, we consider the tolerance interval to be more informative than the (heterogeneous) confidence interval because the first gives the range of treatment effects found by studies while the latter is a confidence interval for a theoretical treatment effect.

In two trials36 ,39 the results for wheeze were not published because the difference between placebo and DSCG was not significant. This could result in some bias in favour of DSCG in the calculation for wheeze. On the other hand, two trials32 ,33 with a positive outcome were omitted from the pooling because the published data were insufficient. The results of the statistical pooling were based on symptoms only because symptom scores were available in all trials.

We did not include studies on the immediate effects of DSCG on exercise induced asthma. This is a different issue which does not pertain to the use of DSCG as advised in the current guidelines. Finally, we did not include studies on the effects of combined treatment—that is, the corticosteroid sparing effects of DSCG.

Given the strong indication of publication bias, the tolerance interval including zero, and the small overall treatment effect, we conclude that, on the basis of published randomised trials, the superiority of DSCG over placebo in the maintenance treatment of children with asthma is not proven. We therefore consider that it is no longer justified to recommend DSCG as a first line prophylactic agent in childhood asthma.