Jump to comment:

• Opioids for the palliation of breathlessness Cochrane review: additional analyses yield same conclusions
  Hayley Barnes, Natasha Smallwood, Julie McDonald and Renee Manser
  Published on: 7 February 2018

• Published on: 7 February 2018

  Opioids for the palliation of breathlessness Cochrane review: additional analyses yield same conclusions

  • Hayley Barnes, Doctor/Researcher Department of Allergy, Immunology and Respiratory Medicine, Alfred Hospital, Melbourne, Australia
  • Other Contributors:
    • Natasha Smallwood, Respiratory Physician/Researcher
    • Julie McDonald, Respiratory Physician/Researcher
    • Renee Manser, Respiratory Physician/Researcher

  We thank Associate Professor Magnus Ekström et al for their research letter regarding our Cochrane Review: Opioids for the palliation of refractory breathlessness in adults with advanced disease and terminal illness (1,2). We also acknowledge that following the publication of their letter in Thorax, feedback was provided through the appropriate mechanism to the Cochrane Review Group (2). We have published a detailed response to their comments in the feedback section of our review, however, given the seriousness of the criticisms published in Thorax, we think it is important that our response also sit alongside their Thorax letter.

  We acknowledge the statistical difficulties in the interpretation and summation of the complex data on opioids for breathlessness. One such issue is the inclusion of crossover studies in a meta-analysis, however, a crossover design is an appropriate way to assess short term interventions, particularly when patient recruitment may be challenging. The Cochrane Handbook outlines several methods to incorporate crossover data into meta-analyses (3). In using the data as if it was a parallel study, the limitations should be acknowledged, in that it can give rise to a unit of analysis error whereby confidence intervals may be wide, and the overall effect is under-estimated. An alternative method is to calculate correlation co-efficients (which describe the ratio of between-patient standard deviation with the within patient variation) to impute...

  Show More

  We thank Associate Professor Magnus Ekström et al for their research letter regarding our Cochrane Review: Opioids for the palliation of refractory breathlessness in adults with advanced disease and terminal illness (1,2). We also acknowledge that following the publication of their letter in Thorax, feedback was provided through the appropriate mechanism to the Cochrane Review Group (2). We have published a detailed response to their comments in the feedback section of our review, however, given the seriousness of the criticisms published in Thorax, we think it is important that our response also sit alongside their Thorax letter.

  We acknowledge the statistical difficulties in the interpretation and summation of the complex data on opioids for breathlessness. One such issue is the inclusion of crossover studies in a meta-analysis, however, a crossover design is an appropriate way to assess short term interventions, particularly when patient recruitment may be challenging. The Cochrane Handbook outlines several methods to incorporate crossover data into meta-analyses (3). In using the data as if it was a parallel study, the limitations should be acknowledged, in that it can give rise to a unit of analysis error whereby confidence intervals may be wide, and the overall effect is under-estimated. An alternative method is to calculate correlation co-efficients (which describe the ratio of between-patient standard deviation with the within patient variation) to impute a corrected standard error. Some included studies provided appropriate data to calculate this (standard error of the differences), or a corrected standard error can be imputed using “borrowed” correlation co-efficients from other studies.

  In our Cochrane Review we used the former method (2). In response to the feedback provided by Ekström et al, we conducted a sensitivity analysis with an alternative meta-analysis (accounting for use of cross over data) using correlation co-efficients and corrected standard errors. The data are presented using standardised mean differences. The results demonstrate a change from baseline SMD -0.14 (95% CI -0.40 to 0.13) and a post treatment score SMD -0.55 (95% CI -0.76 to -0.35) (Figure 1, https://doi.org/10.6084/m9.figshare.5743137). This is similar to our original published results which found a change from baseline score SMD -0.09 (95% CI -0.36 to 0.19) and a post treatment score SMD -0.28 (95% CI -0.5 to -0.05). Both analyses draw the same conclusion that there is a significant but small effect size for the use of opioids for breathlessness.

  Ekström et al raise concerns regarding the use of a fixed effects versus a random effects model (1). Based on the assumption that studies would have a small sample size we chose a priori to use a fixed effects model. As Higgins and Green describe: a random effects model will award relatively more weight to smaller studies because smaller studies are more informative for learning about the distribution of the effects across studies than for learning about an assumed common intervention effect (3). Therefore, if a random effects model is inappropriately applied, in particular, if the results of small studies are systematically different to the results of larger ones, the random effects model can inappropriately exacerbate the effects of any bias (4).

  The choice and rationale for a fixed effects model was outlined in advance in our protocol. This protocol was peer reviewed prior to publication (2). Consistent with Higgins and Green, we presented both a fixed effects and random effects model in the sensitivity analysis, and found no differences in effect (3). Following additional sensitivity analysis as described above, there remains very little difference between the fixed effects model in the change from baseline scores (SMD -0.14 (95% CI -0.40 to 0.13)) and the random effects model (SMD -0.21 (95% CI -0.55 to 0.12)), and in the post treatment score fixed effects model (SMD -0.55 (95% CI -0.76 to -0.35)) and random effects model (SMD -0.69 (95% CI -1.08 to -0.29)).

  A second limitation from the opioids for breathlessness data is the use of different scales to measure the same outcome (e.g. visual analogue scale (VAS) or Borg scale), with scales measured on different lengths, with different extremes, and different gradations of intensity. In order to combine data on different scales, standardised mean differences are required, which are calculated by dividing the mean difference by a pooled estimate of the between-patient standard deviation. However, combining this between-patient standard deviation with the within patient variation imputed from the corrected standard error described as above to incorporate crossover trials is not always possible from the available data. It is difficult to interpret the resulting standardised mean differences from cross-over trials.

  Transforming the data as described above works if the data are reported as either change from baseline or post treatment scores, however it is unclear if it is also appropriate to combine them in a single meta-analysis, and to combine them in a single meta-analysis using standardised mean difference (SMD). Higgins and Green state that post treatment scores can be combined with change from baseline scores when using an unstandardised mean difference, however, they should not be combined as a standardised mean difference using the standard deviation of the change scores (as these are not the same units as the standard deviation of the final scores) (3). Therefore, it makes it difficult to combine data from different scales as outlined above, as well as combining post treatment and change from baseline scores in one single meta-analysis. Originally, we separated post treatment and change from baseline scores. In a further subsequent sensitivity analyses performed in response to the feedback, we combined these but separated by scale, see Figure 2 (https://doi.org/10.6084/m9.figshare.5743134.v2).

  Ekström et al discussed at length the primary outcome of breathlessness, but they did not take into account adverse events or multidimensional assessment of the use of opioids (1). We noted increased adverse events including drowsiness, nausea, and constipation, as well as a significant difference in the mastery domain scores in one included trial, suggesting that participants may feel less in control when using morphine. We believe it is important to consider the evidence in its entirety, rather than focusing on only one effect size score.

  Ekström et al have suggested that we downgraded the quality of evidence based on concerns about study size alone (1). We used GRADE methodology to rate the quality of the evidence and our decision to downgrade the quality of the evidence was based on the fact that more than 50% of included trials did not report on allocation concealment, blinding of participants or personnel, or blinding of outcome assessment. This is potentially a serious limitation when the primary outcome (i.e. change in breathlessness) is entirely subjective. We acknowledge that study size per se does not influence the internal validity of trial results and that some of the trials included in the review were designed with sufficient statistical power.

  The ‘size bias’ criterion was suggested by the Cochrane editorial team during the review process of our manuscript, as there is empiric evidence that study size may be a surrogate marker of trial quality when the reporting on aspects of trial quality is poor (4). In other fields, small study effects have been shown to distort the results of meta-analyses (5). Many of the papers included in the review did not provide sufficient information to adequately assess trial quality, and because all the studies included were small in relative terms (with less than 50 participants per trial) we believe that it is important to highlight that the quantitative data synthesis was based on the pooling of relatively small studies.

  We included the study by Woodcock 1982, but this is more correctly referenced in our review as Bar-Or 1982 (6). We included the study by Johnson et al (2002) in the review, but excluded it from the meta-analyses as the data was not normally distributed and medians and interquartile ranges cannot be imputed into a meta-analysis, consistent with the Cochrane Handbook (3, 7). Although Ekström et al commented that study selection should align to predefined eligibility criteria with reasons for exclusion stated to minimise selection bias, our studies were selected according to a published protocol with study types, inclusion and exclusion criteria specified (1).

  While we value the opinion provided by Ekström et al, the additional sensitivity analyses reported here do not change our review conclusions (1,2). There is some small, low quality evidence that shows benefit for the use of parental or oral opioids to palliate breathlessness in the short term. The magnitude of this benefit is at best modest and given the potential adverse events and the lack of any evidence suggesting an improvement in overall quality of life, longer-term studies with multi-dimensional scales are required to ascertain whether any benefits outweigh the potential long-term risks, particularly where opioids are being used in those with chronic stable disease in the outpatient setting (8).
   
  Acknowledgements
  We thank Christopher Cates for his extensive input on this sensitivity analysis and comments on this letter, Kerry Dwan, Toby Lasserson and the Statistical Methods Group, and Julian Higgins for his report on the interpretation of this data.

  References:
  1. Ekström M, Bajwah S, Bland JM, Currow D, Hussain J, Johnson M. One evidence base; three stories: do opioids relieve chronic breathlessness? Thorax 2017
  2. Barnes H, McDonald J, Smallwood N, Manser R. Opioids for the palliation of refractory breathlessness in adults with advanced disease and terminal illness. Cochrane Database of Systematic Reviews 2016, Issue 3. Art. No.: CD011008. DOI: 10.1002/14651858.CD011008.pub2.
  3. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.
  4. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Internal Medicine 2001;135(11):982-9.
  5. Nüesch E, Trelle S, Reichenbach S, Rutjes AW, Tschannen B, Altman DG, et al. Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study. BMJ 2010;341:c3515.
  6. Bar-Or D, Marx JA, Good J. Breathlessness, alcohol and opiates. The New England Journal of Medicine 1982;306(22):1363–4.
  7. Johnson MJ, McDonagh TA, Harkness A, McKayd SE, Dargie HJ. Morphine for the relief of breathlessness in patients with chronic heart failure--a pilot study. European Journal of Heart Failure 2002;4(6):753–6.

  Figure 1: https://doi.org/10.6084/m9.figshare.5743137 Meta-analysis – Opioids for palliation of breathlessness conducted using alternative methods for use of crossover data with results separated according to post treatment scores and change from baseline scores.

  Figure 2: https://doi.org/10.6084/m9.figshare.5743134.v2 Meta-analysis – Opioids for the palliation of breathlessness using alternative method for use of crossover data, combining post treatment and change from baseline scores but separating by scale used to measure breathlessness.

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top