Article Text
PDF
Abstract
Background The limits of reproducibility of the lung clearance index (LCI) are higher in children with cystic fibrosis (CF) compared with healthy children, and it is currently unclear what defines a clinically meaningful change.
Methods In a prospective multisite observational study of children with CF and healthy controls (HCs), we measured LCI, FEV1% predicted and symptom scores at quarterly visits over 2 years. Two reviewers performed a detailed review of visits to evaluate the frequency that between visit LCI changes outside ±10%, ±15%, ±20% represented a clinically relevant signal. In the setting of acute respiratory symptoms, we used a generalised estimating equation model, with a logit link function to determine the ability of LCI worsening at different thresholds to predict failure of lung function recovery at follow-up.
Results Clinically relevant LCI changes outside ±10%, ±15% and ±20% were observed at 25.7%, 15.0% and 8.3% of CF visits (n=744), respectively. The proportions of LCI changes categorised as noise, reflecting biological variability, were comparable between CF and HC at the 10% (CF 9.9% vs HC 13.0%), 15% (CF 4.3% vs HC 3.1%) and 20% (CF 2.4% vs HC 1.0%) thresholds. Compared with symptomatic CF visits without a worsening in LCI, events with ≥10% LCI increase were more likely to fail to recover baseline LCI at follow-up.
Conclusion The limits of reproducibility of the LCI in healthy children can be used to detect clinically relevant changes and thus inform clinical care in children with CF.
- Cystic Fibrosis
- Lung Physiology
- Paediatric Lung Disaese
- Respiratory Measurement
Data availability statement
Data are available on reasonable request.
Statistics from Altmetric.com
Key messages
WHAT IS ALREADY KNOWN ON THIS TOPIC
The lung clearance index is a promising tool to inform clinical care, but it is currently unclear what defines a clinically meaningful change.
WHAT THIS STUDY ADDS
This study shows for the first time that the biological variability of the LCI is similar in healthy controls and cystic fibrosis (CF).
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE AND/OR POLICY
This study explores thresholds that can be used to identify clinically meaningful changes in LCI in children with CF.
Introduction
The lung clearance index (LCI) is a sensitive physiological outcome measure and is a potentially useful diagnostic tool to guide treatment decisions in children with early cystic fibrosis lung disease (CF).1 2 In a prospective observational study, we showed that LCI is more sensitive than FEV1 in detecting worsening lung function and treatment effects in preschool and school-age children with acute respiratory symptoms.3–6 We also showed that in school-age children with CF, LCI and FEV1 provide complementary information about lung function impairment such that using both measures in combination is superior to using one measure alone for clinical monitoring.
However, it remains unclear what constitutes a clinically meaningful change in LCI between test occasions.1 7 Establishing diagnostic thresholds for an outcome measure is essential to make results directly interpretable to patients and clinicians.8 There are different ways to define a clinically significant change; the first is to understand a test’s biological variability, such that a change that is larger than this threshold can be considered relevant. The second approach is to quantify the ability of an outcome measure to predict clinical outcomes of interest.9
The biological variability of LCI in healthy children and in children with CF has been described previously. Longitudinal observational studies in healthy children have shown that LCI changes by ±15% between test occasions.5 10 In CF, the variability between tests is higher for measurements taken 1–3 months apart (17%–25%) compared with those taken 24 hours apart (±15%).5 11 12 This suggests that some of the increased variability in people with CF reflects changes in disease state rather than test variability.13
This study aims to evaluate the thresholds in children with CF that could facilitate the interpretation of LCI in clinical practice. Using data from a prospective observational study in school-age children with CF and healthy controls, we evaluated the frequency that LCI changes outside ±10%, ±15%, ±20% represent a clinically relevant signal. In the setting of acute respiratory symptoms, we also investigated the ability of LCI worsening at different thresholds to predict failure of lung function recovery at follow-up.
Methods
This is a secondary analysis of a prospective longitudinal study in school-age children with CF and healthy controls that was conducted at the Hospital for Sick Children, Toronto (REB no: 1000055762) and Riley Hospital for Children, Indianapolis (REB no: 1702210252).6 10 The ethics committee of each institution approved the study, and parents gave written informed consent to participate and assent was obtained from participants where appropriate.
Children were included if they had a confirmed diagnosis of CF as defined by two or more clinical features of CF and a documented sweat chloride of >60 mEq/L by quantitative pilocarpine iontophoresis test or a genotype showing two pathogenic mutations. Participants were excluded if they had undergone previous organ transplantation or had chronic lung disease not related to CF. Healthy control participants were previously enrolled in a preschool longitudinal observational study (<6 years) at the Hospital for Sick Children, Toronto, and the Riley Hospital for Children, Indianapolis,3 4 and were reconsented into this follow-on study (5–10 years). Children with chronic lung diseases such as asthma were excluded from the study. Further study details of the eligibility criteria and study procedures are published elsewhere.6 10
Participants attended a study visit every 3 months in keeping with routine clinical follow-up, and during episodes of increased respiratory symptoms, for 2 years. At each study visit, a clinical history and physical examination were conducted by a CF physician. The visit was classified as stable if the participant was judged to be at their baseline clinically. The visit was classified as symptomatic, or as an acute respiratory event, if a participant’s symptoms were increased compared with a participant’s baseline, regardless of whether treatment was administered. All clinical and research documentation was reviewed (by LP and FR) when categorising study visits, as outlined in detail in a previous publication.6 LCI measurements were not considered during the process.
Study outcomes
Lung function
Nitrogen multiple breath washout (MBW) was performed at every visit using the Exhalyzer D (Ecomedics, Duernten, Switzerland; Spiroware software V.3.3.1) device following standardised protocols based on the MBW consensus statement.14 Earlier Spiroware software versions were used during data collection, whereas the LCI values reported were derived using the latest Spiroware V.3.3.1, which corrected for systematic errors seen with previous software versions.15
Spirometry was performed according to American Thoracic Society and European Respiratory Society guidelines.16 Per cent-predicted values were calculated using the Global Lung Function Initiative reference equations.17 Treating clinicians were blinded to LCI measurements during the study.
Patient-reported outcomes
Age-appropriate versions of the Cystic Fibrosis Questionnaire—Revised (Respiratory Domain only) were completed by CF participants (version for children aged 6–13 years or ≥14 years) and parents (for children aged ≤13 years) at each visit. Responses of this six-item domain were standardised from 0 to 100, with higher scores indicating better health-related quality of life.18 The Chronic Respiratory Infection Symptom Score (CRISS) was an additional symptom score completed by CF participants with higher scores indicating more severe respiratory symptoms.19 The CRISS is validated for patients 12 years and older but was administered to all CF participants in the study. Symptom scores were collected by study investigators but were not shared with clinicians or used to inform clinical care.
Definition of clinically relevant changes
We identified all visits, both CF and healthy control visits, at which LCI increased or decreased by at least 10% relative to the last study measurement. Calculating changes relative to the last measurement allowed us to include data from all study visits (except the first visit) and facilitated a direct comparison between the CF and healthy control cohorts. As a sensitivity analysis, we also calculated LCI changes relative to the last stable visit; if the last consecutive study visit was not clinically stable, then lung function measurements from the second last visit were taken as the baseline.
One reviewer (LP) initially categorised the LCI change as either signal (ie, clinically relevant change), noise (ie, the biological variability of the test) or uncertain. Two reviewers (LP and FR) then evaluated the data together; disagreements were discussed, and consensus was achieved in all cases.
To minimise bias in our decision making, we used the following predetermined factors to label LCI changes as a signal: (1) there was a clinical event, for example, LCI increased at a visit with acute respiratory symptoms or LCI decreased after starting a treatment such as antibiotics, cystic fibrosis transmembrane conductance regulator (CFTR) modulators or dornase alfa; (2) the change was consistent with changes in respiratory symptoms, as reported to the clinical and/or research team or suggested by a corresponding improvement or worsening in symptom scores between visits; (3) there was a corresponding change in FEV1 and/or (4) the change in LCI was sustained at the next study visit. LCI changes were labelled as noise if they did not meet the above criteria for signal and the reviewers were satisfied that the change represented biological variability. If the reviewers could not confidently label an LCI change as either signal or noise, it was labelled as being of uncertain significance.
We performed a descriptive analysis of the proportion of CF and healthy control visits with LCI changes outside ±10%, ±15% and ±20%. We describe the proportions of LCI changes at each threshold labelled as ‘signal’, ‘noise’ and ‘uncertain’. Given that the LCI changes in the ‘uncertain’ category did not meet our predetermined criteria for clinical signal and may represent biological variability, the LCI changes labelled as ‘uncertain’ are presented on their own and together with the ‘noise’ category. The thresholds of ±15% and ±20% were chosen to reflect the 95% limits of reproducibility in healthy children and CF, respectively.5 12 20 We previously showed that the visit-to-visit variability of LCI and FEV1 are similar even though a 10% drop in FEV1 is normally considered clinically relevant.10 We, therefore, also evaluated LCI changes using the ±10% threshold.
LCI to predict lung function recovery after acute respiratory events
To account for repeated measurements in the same participant, we used a generalised estimating equation model, with a binomial family, and a logit link function and an exchangeable correlation structure to determine whether CF participants with acute respiratory symptoms and an increase in LCI above a specific threshold (≥10%, ≥15% or 20%) were more likely than symptomatic participants with stable LCI values to fail to recover ≥90% baseline lung function (LCI and FEV1) at the next follow-up visit to determine whether CF participants with acute respiratory symptoms and an increase in LCI above a specific threshold (≥10%, ≥15% or 20%) were more likely than symptomatic participants with stable LCI values to fail to recover ≥90% baseline lung function (LCI and FEV1) at the next follow-up visit. Only acute respiratory events with a stable baseline and a follow-up visit were included in this analysis. For all thresholds recovery was defined as returning to ≥90% baseline lung function. Baseline for both LCI and FEV1pp (FEV1 per cent predicted) was defined as the most recent measurement from a stable visit before an acute respiratory event. If the study visit directly before the symptomatic visit was not clinically stable, then lung function measurements from the second last visit were taken as the baseline. If neither the last nor the second last study visits were clinically stable, then there was no baseline for that respiratory event and it was excluded from the analysis. In a sensitivity analysis, we repeated the analysis using an alternate definition of baseline lung function, the lowest (ie, best) LCI or highest FEV1 values in the last 6 months.
Respiratory events associated with treatment with antibiotics are typically associated with more severe clinical features6; therefore, these events were analysed separately.
Results
Study population
Eight hundred and sixty-four CF study visits from 98 participants and 343 healthy control visits from 48 participants were captured in this study (figure 1). Participants were followed for a mean (range) of 2.0 (0.3–2.2) years. Visits with a previous LCI measurement to calculate a change were available at 744 CF visits and 293 healthy control visits. The demographics of the study population are shown in table 1.
Flow diagrams of study population; numbers in brackets are %. CF, cystic fibrosis; LCI, lung clearance index.
Demographics of study population at enrolment
Comparison of LCI changes in children with CF and healthy controls
The proportions of visit-to-visit LCI changes outside ±10%, ±15% and ±20% for children with CF and healthy controls are shown in table 2.
Proportions of LCI changes labelled as clinically relevant ‘signal’, ‘noise’ or of ‘uncertain’ significance for children with cystic fibrosis and healthy controls
In healthy controls, a relative LCI change of ±15% was observed for 4.4% (n=13) of visits (table 2). Of the four healthy control visits that were labelled as a signal, three had LCI changes that reflected worsening in respiratory status with documented acute respiratory symptoms at the visit and one reflected resolution of respiratory symptoms since the last visit. By comparison, 19.4% of the children with CF had visit-to-visit LCI changes of at least ±15% (n=144), 77.8%% of which were labelled as signal (table 2). The specific reasons for labelling the LCI changes in CF children as a signal are shown in table 3. The presence of acute respiratory symptoms was the most common reason for a 15% increase (worsening) in LCI between study visits (55/66; 83.3%). Response to a newly initiated therapy was the most common reason for a 15% decrease in LCI between study visits (38/46; 82.6%). In terms of specific initiated treatments, antibiotic therapy was the most common (n=28) followed by dornase alfa (n=6) and CFTR modulators (n=4).
Reasons for labelling LCI changes as clinically relevant signal
At the ±15% threshold, the proportion of LCI changes categorised as noise, reflecting biological test variability, was similar in children with CF (3.4%) and healthy controls (3.1%) (table 2). The proportion of LCI changes labelled as noise was also comparable between health and CF at the ±10% and ±20% thresholds (figure 2).
Proportion of all visits, cystic fibrosis (CF) (n=744) and healthy controls (HC) (n=293), with a ±10%, ±15% and ±20% increase or decrease in lung clearance index (LCI) relative to the last visit.
Comparison of LCI changes outside ±10%, ±15% and ±20% in children with CF
In CF, compared with the ±15% threshold, decreasing the threshold to ±10% increased the proportion of clinically relevant signal (25.7%) but also increased the proportion of noise (6.9%) (table 2). Increasing the threshold to ±20% had the opposite effect, decreasing the identification of both signal (8.3%) and the noise (1.9%). However, as shown in figure 2, at all thresholds the relative proportion of signal and noise was similar, such that 72.0%, 77.8% and 77.5% of LCI changes at the ±10%, ±15% and ±20% thresholds, respectively, were labelled as clinically relevant signal.
LCI to predict lung function recovery following acute respiratory events
There were 281 acute respiratory events captured during the study, of which 213 (137 treated and 76 untreated with antibiotics) had both a recent stable baseline visit and a follow-up study visit. Of the 137 acute respiratory events treated with antibiotics, 44.5% (n=61), 25.6% (n=35) and 19.0% (n=26) were associated with an LCI increase ≥10%, ≥15% and ≥20%, respectively (table 4). Of the 76 acute respiratory events that were not treated with antibiotics, 26.3% (n=20), 18.4% (n=14) and 11.8% (n=9) were associated with an LCI increase ≥10%, ≥15% and ≥20%, respectively.
A generalised estimating equation model, with a logit link function to investigate whether CF participants with acute respiratory symptoms and an increase in LCI above a specific threshold (10%, 15% or 20%) were more likely than symptomatic participants with stable LCI values to fail to recover ≥90% baseline lung function (LCI and FEV1) at the next consecutive study visit
Acute respiratory events associated with ≥10% increase in LCI (reference group acute respiratory events associated with LCI change <10%) were more likely to fail to recover baseline LCI (OR 2.4, 95% CI 1.4 to 4.2; p=0.002) and FEV1 at follow-up (OR 2.3, 95% CI 1.1 to 4.8; p=0.02) (table 4). ORs were consistent when the analysis was repeated using higher thresholds for LCI change (≥15% and ≥20%). In the stratified analysis, visual comparison of acute respiratory events treated with antibiotics and untreated events suggest that untreated events with a worsening in LCI ≥10% were more likely to fail to recover baseline LCI at follow-up (table 4). However, the analysis was underpowered to investigate this in more detail.
Sensitivity analyses
Online supplemental table S1 shows the proportions of LCI changes labelled as ‘signal’, ‘noise’ and ‘uncertain’ when the last stable visit rather than the last consecutive visit was used to calculate a change. The proportion of ‘noise’, representing biological variability, was similar in the CF and healthy control groups. At the 15% threshold, 69.3% (70/101) of LCI changes represented signal, compared with 77.8% (112/144) using the last consecutive visit as the comparison visit. Calculating changes relative to the last stable visit rather than the last consecutive visit reduced the proportion of signal attributable to a treatment effect (38/112; 33.9% vs 8/70; 11.4%). This can be explained by antibiotic treatment effects between symptomatic and consecutive follow-up visits not being captured (online supplemental table S2).
Supplemental material
When the prediction analysis was repeated using an alternate definition of baseline lung function—the lowest (ie, best) LCI or highest FEV1pp in the previous 6 months—the results were consistent with the main analysis (see online supplemental table S3). We also repeated the analyses for each site separately and, again, the results were consistent with the overall analysis (see online supplemental tables S4, S5).
Discussion
The data presented in this study confirm that the 15% threshold for LCI is reasonable to detect clinically meaningful changes in CF. LCI detected a higher proportion of signal when a threshold of 10% was used, but this comes with an increase in noise and should be balanced with the pretest probability of a clinical signal for an individual (eg, if they are symptomatic).
We observed LCI changes outside 15% in 4.4% of healthy control visits and 19.4% of CF visits. However, a detailed review of the clinical data showed that only 4.3% of the LCI changes outside 15% at CF visits were noise. Lowering the LCI threshold to 10% increased the detection of clinically relevant changes; the noise of the test doubled but remained relatively low and comparable to healthy controls. Increasing the LCI threshold to 20% resulted in the identification of considerably less clinically relevant changes. Taken together, the results of this study suggest that the biological variability of LCI is similar in healthy children and CF.
Thresholds for diagnostic tests provide a basis for deciding whether to treat, do more testing or do nothing.21 In this study, we did not calculate sensitivity and specificity and so did not define a specific cut-off for LCI change. However, for many diagnostic tests, there is not a single threshold for clinical-decision making; there are multiple potential thresholds at which different clinical decisions can be made, often reflecting clinical uncertainty.8 Despite FEV1 being the established monitoring tool to guide clinical decision-making in CF, thresholds to define clinically meaningful changes have not been established. A drop of 10% is usually considered clinically meaningful even though a change of this magnitude is within the variability of the test (and similar to LCI).10 In the clinical setting, it is quite clear that changes within the limits of reproducibility of FEV1 (ie, <15%) are also relevant. The interpretation of changes in LCI between visits depends on the overall clinical context and the pretest probability of a meaningful change in respiratory health. For example, if a patient presents to clinic with acute respiratory symptoms, a 10% worsening in LCI would be sufficient to trigger treatment with antibiotics. By contrast, if a patient is clinically at their baseline, a deterioration in LCI of 10% may not necessarily trigger a change in treatment, but given that the rate of LCI change in children with CF is an important predictor of future lung function,10 the clinician may want to repeat the test at follow-up to ensure that the LCI has returned to baseline.
Pulmonary exacerbations are important clinical events in the lives of individuals with CF and directly contribute to the progression of lung disease.22–25 In this cohort, we previously described the trajectories of LCI in early school-age years and identified that treatment with either oral or intravenous antibiotics was associated with higher LCI at visits where these factors were present.10 The recovery of FEV1 after pulmonary exacerbations has been well studied,23 26–28 but relatively little is known about LCI recovery. We previously reported that 25% of school-age children with CF do not recover ≥90% baseline LCI after mild acute respiratory events.6 In this study, we show that a worsening in LCI ≥10% increased the odds of failing to recover baseline lung function in children with acute respiratory symptoms, as measured by both LCI and FEV1. These results support using a threshold of 10% for LCI worsening to guide treatment decisions in symptomatic children with CF.
Before LCI is integrated into clinical care, a robust assessment of its ability to impact decision making and improve patient outcomes is required. Clinical utility should be considered within the context of the additional time and resources required to carry out this test. Considering that LCI remains stable over time for most children with mild CF lung disease,7 10 29 30 it is unlikely to provide useful information about every patient at every clinic visit. Thus, further work is needed to determine the specific circumstances when LCI has the potential to add value clinically. Horsley and coworkers recently published a prospective observational study investigating the longitudinal change in LCI in 112 children and adults with CF; in a subset (n=27) of the cohort, LCI provided additional information about a subject’s clinical status and informed clinical decision making in 14 (52%).7 LCI could be used to promote individualised treatment strategies by distinguishing the children with low and stable LCI who could be managed with minimal intervention from those who would benefit from more intensified CF care.
Strengths and limitations
A strength of this study was that the LCI measurements were not shared with physicians or used to guide clinical care. In addition, we collected data from healthy controls which provide a valuable comparison in our analysis. However, this study also has some limitations. Two reviewers performed a detailed and standardised review of individual visits to mitigate the subjectivity involved in reviewing multiple pieces of clinical data, but the labels assigned required clinical judgement. We used an ‘uncertain’ category for the small number of visits where a determination of signal or noise could not be made.
We did not capture visits with clinically relevant changes in symptoms and/or FEV1 that did not have a change in LCI. LCI was the focus of this study as symptoms and FEV1 are established clinical signals they were outside the scope of this study. Finally, this study was conducted before the widespread introduction of triple combination modulator therapy. Highly effective modulator therapy may impact the magnitude of visit-to-visit LCI changes but should not impact the biological variability.
In summary, the increased variability of LCI in children with CF compared with healthy controls is likely due to clinically relevant changes in respiratory status rather than biological variability.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by the ethics committee of each institution. Hospital for Sick Children, Toronto (REB no: 1000055762) and Riley Hospital for Children, Indianapolis (REB no: 1702210252) approved the study and parents gave written informed consent to participate and assent was obtained from participants where appropriate. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
The authors thank the children and their families for their participation in this research study. Preliminary results from this study were presented in abstract form at the European Respiratory Congress (2021) and the North American Cystic Fibrosis Conference (2021).
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @luperr84
Contributors LP contributed to conceptualisation of the study design, data collection, data analysis, data interpretation, figures and wrote the manuscript. SS contributed to conceptualisation and design of the study, funding acquisition, data analysis, data interpretation, figures and manuscript review and editing; SDD contributed to funding acquisition, study design, data interpretation and manuscript review and editing. MS, HG, VJW, NS and DS contributed to study design, data interpretation and manuscript review and editing. FR conceptualised, designed and supervised the study and contributed to funding acquisition, data analysis, data interpretation, figures, manuscript review and editing, and is the guarantor of the study.
Funding Funding for the study was obtained from the Cystic Fibrosis Foundation (award no RATJEN16A0), Cystic Fibrosis Canada (Grant ID: 3187) and the Cystic Fibrosis Foundation Therapeutics Development Network National Resource Centre (Center for Pediatric Lung Function). LP is a PhD candidate with the Royal College of Surgeons in Ireland and this research was funded by the National Children's Research Centre, Crumlin (Grant No. D/19/4).
Competing interests SS reports grants from Cystic Fibrosis Canada and US Cystic Fibrosis Foundation, during the conduct of the study. MS reports grants from Vertex Pharmaceutical and Mylene Pharmaceutical, payment for educational module from Vertex Pharamaceutical, outside the submitted work. FR reports grants and personal fees for consultancy from Vertex, Proteostasis, Genentech, Bayer, Novartis and Roche outside the submitted work. The other authors have nothing to disclose.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.