• Idiopathic pulmonary fibrosis

It is, perhaps, an embarrassment of riches for idiopathic pulmonary fibrosis (IPF) researchers and providers—the approval of pirfenidone and nintedanib for the treatment of IPF and on its heels, an unprecedented interest on the part of pharmaceutical companies to sponsor clinical trials to test new candidates. In addition, large amounts of clinical data from patients with IPF of very diverse genetic and environmental backgrounds accompanied the clinical trials for pirfenidone and nintedanib. And the new studies promise to yield even more data, repositories of blood, cells, DNA and RNA, at an ever-increasing degree of sophistication. With the availability of these biorepositories and data repositories, very provocative questions can be asked—and potentially answered.

It is in this context that we read in Thorax a new post hoc analysis by Professor Kreuter and colleagues of 624 patients with IPF pooled from the placebo arms of the pirfenidone trials (CAPACITY 004 and 006 and ASCEND).1 The authors have previously analysed these patients and have addressed some intriguing questions, including how anticoagulation and acid suppression in IPF for comorbid conditions affected study outcomes.2 ,3 In the present article, they have turned their attention to another question that has dogged clinical researchers previously, namely how the use of 3-hydroxy-3-methyl-glutaryl-coenzyme A reductase (HMG coenzyme A) reductase inhibitors, popularly known as ‘statins’, affects outcomes in IPF. The authors employed a shared frailty model, which is an extension of the Cox regression model,1 to determine the effect of statins on study outcomes without adjustment and a multivariate model to adjust for age, sex, smoking, pulmonary function, medical history, dyspnoea, history and cardiovascular risk factors.1 The authors found after unadjusted analyses that statins had no effect on mortality, change in pulmonary function, change in 6 min walk distance and all-cause hospitalisation. After multivariate analysis, however, the authors found that users of statins exhibited significantly reduced risk of death, all-cause hospitalisation and IPF-related mortality. The results of this study represent a positive response to the very murky results from other studies (some of which are post hoc analyses) that report that statins protect patients from the loss of pulmonary function in ageing men4 and may reduce interstitial lung disease (ILD) mortality5 but may also worsen ILD mortality or even predispose to interstitial lung abnormalities.6 ,7 Furthermore, statins may simply have neither beneficial nor detrimental effects.8 ,9

Exciting indeed on first glance, this represents a longitudinal study of one of the largest and best-phenotyped IPF cohorts. As the authors note, the statin users were older and had a higher prevalence of comorbid cardiovascular disease—a population that you would expect to have an enhanced risk of mortality. After all, Professor Kreuter has previously reported that cardiac disease (but not diastolic dysfunction) significantly increases mortality in IPF.10 So do statins have an independent and specific effect on pulmonary fibrosis, above and beyond their effect on cardiovascular disease? It's hard to say with any certainty. The authors did not detect an effect of statins on changes in FVC or diffusing capacity for carbon monoxide, both well-validated measures of outcomes in IPF.11 ,12 Perhaps the beneficial effects of statins can be explained by other confounders not measured and not available to the investigators, such as a higher socioeconomic status or better access to care? The benefit of statins in these patients may be an example of bias associated with more attentive care.

Then, of course, is the difficulty of interpreting favourable p values observed during subgroup analyses, and in particular post hoc analyses, where the hypotheses were not specified prior to examination of the data. The risks of subgroup analyses and multiple testing have been described previously.13–15 Multiple testing is well known to inflate the risk of randomly discovering statistically significant associations. Replication of positive findings from post hoc analyses has been notoriously difficult, so acting on the results of post hoc analyses becomes particularly problematic. Our experience in IPF with retrospective interpretation of clinical trials has been particularly vexing. On the one side, much of what we know about the natural progression of disease comes from the placebo arm of an early trial,16 but on the other hand, the field had suffered from misplaced faith in post hoc analyses. As an example, the first interferon (IFN)-γ placebo-controlled randomised clinical trial was essentially negative.17 However, post hoc analysis of the data suggested a benefit of IFN-γ-1b in a subset of patients with IPF,17 leading to design of a larger trial focusing on this subgroup. This trial was, as is well known, stopped after the second interim analysis for lack of efficacy.18 Similarly, interpretation of a study missing a placebo arm19 led to adoption of a presumably harmless therapeutic strategy later proven to be detrimental to patients with IPF.20 And now, with the completion of this wave of successful studies, we see many secondary analyses that could have potentially actionable implications.

So, how should the medical community respond? In our view, there are several layers to the response. Of primary importance is the impact of these studies on practising clinicians. These observations should not be considered endorsement of statin therapy for the treatment of patients with IPF. Our collective experience in IPF suggests that it is better for us to err on the side of caution for even the most harmless interventions may change the delicate balance of the lung in unpredictable ways. Thus, statins, proton pump inhibitors and whatever else are highlighted by secondary analyses, should be prescribed based on their primary indication and not based on a ‘notion’ of benefit or harm. Second, those performing and reporting these secondary analyses should make an effort to validate the results by obtaining access to information from other arms from the same studies or from other studies. Because their results may have significant operational implications, we think it is their obligation to complete the task they have embraced and validate their results. And finally the community at large, researchers, clinicians and pharmaceutical companies must commit to sharing data from interventional trials. This year, the International Committee of Medical Journal Editors (ICMJE) recommended that authors of published clinical trials will be required to share de-identified subject data within 6 months of publication. One could only imagine what would be the effect of such a policy. Instead of multiple observations based on subanalyses of parts of a study, we will have access to multiple full data sets. This will allow other investigators to apply their own unique analyses and determine whether the results are reproducible. One existing approach to data sharing is the Yale Open Data Access (YODA) Project.21 YODA acts as an independent but trusted third party that facilitates fair and responsible access to participant-level clinical trial data as well as full clinical study reports. Medtronic, Johnson & Johnson and SI-BONE have partnered with YODA and the website (http://yoda.yale.edu/) lists 18 studies. We believe that post hoc analyses will allow reproducible and potentially clinically actionable conclusions only when the complete clinical data (including treatment arms) from all interventional studies in IPF will be available for parallel analysis by multiple groups. Thus, whether or not statins are beneficial in the treatment of patients with IPF, we cannot say for certain. But, we applaud the authors of this paper for their effort to raise the question, and hopefully, they will also join us in our call to all investigators that all clinical trial data be made publicly available to all.