Article Text

PDF

Original article
Validation of a health-related quality of life instrument for primary ciliary dyskinesia (QOL-PCD)
  1. Laura Behan1,2,3,4,
  2. Margaret W Leigh5,
  3. Sharon D Dell6,7,8,9,
  4. Audrey Dunn Galvin4,
  5. Alexandra L Quittner10,
  6. Jane S Lucas1,2,3
  1. 1 Primary Ciliary Dyskinesia Centre, University Hospital Southampton NHS Foundation Trust, Southampton, UK
  2. 2 NIHR Southampton Respiratory Biomedical Research Unit, University of Southampton and University Hospital Southampton NHS Foundation Trust, Southampton, UK
  3. 3 Academic Unit of Clinical and Experimental Sciences Faculty of Medicine, University of Southampton, Southampton, UK
  4. 4 School of Applied Psychology, University College Cork, Cork, Ireland
  5. 5 Department of Pediatrics and Marsico Lung Institute, University of North Carolina School of Medicine, Chapel Hill, North Carolina, USA
  6. 6 Division of Respiratory Medicine, The Hospital for Sick Children, Toronto, Ontario, Canada
  7. 7 Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Ontario, Canada
  8. 8 Department of Pediatrics, University of Toronto, Toronto, Ontario, Canada
  9. 9 Institute of Health Policy, Management and Education, University of Toronto, Toronto, Ontario, Canada
  10. 10 Department of Psychology, University of Miami, Coral Gables, Florida, USA
  1. Correspondence to Dr Jane S Lucas, Faculty of Medicine, Mail Point 803, University Hospital Southampton, Southampton SO16 6YD, UK; jlucas1{at}soton.ac.uk

Abstract

Background Quality of life (QOL)-primary ciliary dyskinesia (PCD) is the first disease-specific, health-related QOL instrument for PCD. Psychometric validation of QOL-PCD assesses the performance of this measure in adults, including its reliability, validity and responsiveness to change.

Methods Seventy-two adults (mean (range) age: 33 years (18–79 years); mean (range) FEV1% predicted: 68 (26–115)) with PCD completed the 49-item QOL-PCD and generic QOL measures: Short-Form 36 Health Survey, Sino-Nasal Outcome Test 20 (SNOT-20) and St George Respiratory Questionnaire (SGRQ)-C. Thirty-five participants repeated QOL-PCD 10–14 days later to measure stability or reproducibility of the measure.

Results Multitrait analysis was used to evaluate how the items loaded on 10 hypothesised scales: physical, emotional, role and social functioning, treatment burden, vitality, health perceptions, upper respiratory symptoms, lower respiratory symptoms and ears and hearing symptoms. This analysis of item-to-total correlations led to 9 items being dropped; the validated measure now comprises 40 items. Each scale had excellent internal consistency (Cronbach's α: 0.74 to 0.94). Two-week test–retest demonstrated stability for all scales (intraclass coefficients 0.73 to 0.96). Significant correlations were obtained between QOL-PCD scores and age and FEV1. Strong relationships were also found between QOL-PCD scales and similar constructs on generic questionnaires, for example, lower respiratory symptoms and SGRQ-C (r=0.72, p<0.001), while weak correlations were found between measures of different constructs.

Conclusions QOL-PCD has demonstrated good internal consistency, test–retest reliability, convergent and divergent validity. QOL-PCD offers a promising tool for evaluating new therapies and for measuring symptoms, functioning and QOL during routine care.

  • Rare lung diseases
  • Respiratory Measurement
  • Psychology

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Key messages

What is the key question?

  • This work evaluates the performance of the first health-related quality of life measure for adults with primary ciliary dyskinesia (QOL-PCD), to overcome the lack of a reliable outcome measure in this illness.

What is the bottom line?

  • Psychometric validation has shown that QOL-PCD is a reliable and valid patient-reported outcome measure for adults with PCD.

Why read on?

  • This work has led to the first validated health-related QOL instrument specific for PCD, providing a promising outcome measure for use in clinical trials and clinical practice.

Introduction

Primary ciliary dyskinesia (PCD) is a rare, heterogeneous genetic disorder characterised by impaired mucociliary clearance due to abnormal ciliary function. Individuals with PCD usually present with unexplained neonatal respiratory symptoms in the first few days of life,1 ,2 have early onset of persistent sinopulmonary infections, bronchiectasis during childhood3 and a progressive decline in lung function.4 This can lead to end-stage lung disease with a report that 25% of adult patients with PCD in the USA required long-term oxygen or lung transplantation.5 Male infertility is common since sperm flagella have a similar ultrastructure to cilia; the incidence of female infertility and of ectopic pregnancy is unclear and is explained by immotile fallopian tube cilia. Motile embryonic nodal cilia contribute to left-right asymmetry and nearly half of patients with PCD exhibit situs inversus, and 12% heterotaxy syndromes, sometimes associated with complex congenital cardiac defects.6

Outcome measures that have been used to assess disease severity in PCD include spirometry,4 chest CT,3 ,7 MRI8 and lung clearance index.9–12 These physiological and radiological measures all have limitations in terms of their sensitivity or feasibility to monitor disease progression. The Food and Drug Administration (FDA) and the European Medicines Agency strongly endorse the use of outcome measures in clinical trials, assessing the impact of the disease on the patient's daily symptoms and functioning (eg, physical, respiratory, social) in addition to physiological measures.13–15 Treatment strategies for PCD have necessarily been applied from other diseases,16 ,17 particularly cystic fibrosis, since no medications have specifically been tested and approved for PCD. A major obstacle to evaluating new treatments and monitoring disease progression is the lack of a disease-specific outcome measure.18

Thus, we developed health-related quality of life (HRQOL) measures to assess the impact of PCD for children, teenagers and adults from the patient perspective. 19 Here, we present the psychometric validation of the English version of QOL-PCD in adults from the UK, the USA and Canada.

Methods

Participants

Participants were recruited from PCD diagnostic centres across the UK, the USA and Canada. Adults (aged ≥18 years) with a positive diagnosis of PCD were eligible to participate. Information about the study was provided at a clinic appointment or by telephone. The following inclusion criteria had to be met by patients: (1) diagnosis of PCD in one of the specified diagnostic centres, (2) age ≥18 years and (3) ability to read and speak English fluently. The UK participants had been diagnosed at one of the English diagnostic centres20 ,21 based on clinical phenotype plus high-speed video analysis of ciliary function and/or assessment of ciliary ultrastructure by electron microscopy. North American participants were diagnosed at a specialised PCD research centre, based on: a compatible clinical phenotype plus defect in ciliary ultrastructure and/or identification of biallelic disease-causing mutations in one of the PCD genes.

QOL-PCD scales

The QOL-PCD adult version consisted of 49 items,19 which were self-completed electronically at home or in the clinic, where possible; ‘pen and paper’ copies were provided to those without access to internet. Development of QOL-PCD followed the procedures and analyses recommended by the FDA guidance on patient-reported outcome measures.15 Previously reported content validity, clinical relevance score and cognitive testing conducted on interim versions supported QOL-PCD concepts, items and scale options.19 Participants were provided with a unique study number and a link to the online survey. No identifiable information was included, and the data were captured on a server of University of Southampton. Most responses used a 4-point Likert scale: ‘not at all true’ to ‘very true’ or ‘never’ to ‘always’. The first time QOL-PCD was completed, participants also completed generic questionnaires: Short-Form 36 Health Survey (SF-36), the shortened St George Respiratory Questionnaire (SGRQ-C) and a measure focusing on rhinosinus symptoms: Sino-Nasal Outcome Test 20 (SNOT-20). Further details on this protocol and these measures can be found in the online supplementary data.

Statistical analysis

Statistical analyses were conducted using the SPSS Statistics for Windows, V.21.0 (IBM, Armonk, New York, USA); p<0.05 was considered to be statistically significant.

To estimate the sample size needed to calculate internal consistency, we used equations developed by Bonett22 to identify sample sizes needed to establish Cronbach's α. This formula used parameters for precision, number of items and level of reliability. For an average of five items per scale, a sample size of 59 yielded an α coefficient of 0.70 with 95% CI.

We assessed the distribution of responses for each item and each scale to look for floor and ceiling effects. We conducted multitrait analysis to test the fit between items and their hypothesised versus competing scales. This type of analysis was developed for smaller samples for which factor analysis is not appropriate. These analyses assessed the extent to which items correlated with their hypothesised versus competing scales; we required item-to-scale correlations ≥0.40 with the intended scale and lower correlations with competing scales.23–25 We considered floor and ceiling effects, using <15% of participants as the threshold for the highest and lowest scores for a scale.26

Reliability

Internal consistency of the QOL-PCD scales were investigated by Cronbach's α values. Cronbach’s α gives a score of between 0 and 1, a value of >0.70 indicating good internal consistency. Items were removed if this led to higher internal consistency (ie, higher Cronbach's α) to increase the parsimony and efficiency of the instrument.27 ,28 The distributions of responses and multitrait analyses were reviewed in a series of teleconferences to decide which items could be removed, allowing us to shorten QOL-PCD, taking reliability and clinical relevance into consideration.

Test–retest reliability was assessed using intraclass correlation coefficients (ICC) in stable patients who completed the QOL-PCD a second time, 10–14 days after completing the baseline measures. An ICC value of >0.60 provided evidence of good stability and >0.75 excellent stability for each scale.

Validity

For a questionnaire to be construct valid, all items together should represent the underlying construct (HRQOL). Construct validity can be determined by testing the instrument against hypotheses. We hypothesised a priori that clinical features (age, gender, FEV1 and Pseudomonas status) would correlate with specific scales (eg, FEV1 would correlate with physical functioning and upper respiratory symptoms (construct validity). Cohen's guidelines for the interpretation of correlation coefficients were used; correlations between 0.50 and 1.00 were interpreted as strong, correlations between 0.30 and 0.50 as moderate, correlations between 0.10 and 0.30 as small and correlations <0.1 as weak.29 We also predicted that QOL-PCD scales would have moderate correlations (>0.3), using Spearman's correlation, with generic scales (SF-36, SGRQ-C and SNOT-20) measuring similar constructs (convergent validity). We hypothesised small or weak correlations (<0.3) with scales measuring different constructs (divergent validity). Details of the hypothesised correlations are provided in tables 4 and 5.

Distribution-based measures of clinical significance

Minimal clinically important difference (MCID) in a HRQOL score is defined as ‘the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient's management’.30 MCID can be determined using anchor-based and distribution-based methods. Distribution-based methods rely on the statistical distributions of HRQOL scores in a given study. We applied two distribution-based methods in this study. These were: (1) SDs of QOL-PCD scale mean scores were divided by 2 to establish a 1/2 SD change and (2) the SE of measurement (SEM) was generated for all QOL-PCD scores using the following formula: SD√(1−α) (SD=SD of mean baseline QOL-PCD scale score and α=scale reliability).31 Wyrwich et al 32 suggested that a difference of 1 SEM frequently corresponds to a MCID.

Ethical approval

The study was approved by the National Research Ethics Service, UK (UK 07/Q1702/109), the Research Ethics Board at the Hospital for Sick Children in Toronto, Canada and the Institutional Review Boards at the University of North Carolina, Chapel Hill. Written consent was obtained prior to participation.

Results

Participants

Between April 2014 and March 2016, 72 adults were recruited, slightly more than that required to establish reliability. Participant characteristics are shown in table 1. The 49-item prototype took a mean time of 8 min (SD=5) to complete.

Table 1

Participant characteristics by country of residence

Development of scales

We used a multitrait analysis to generate 10 hypothesised scales: physical, emotional, role and social functioning, treatment burden, vitality, health perceptions, upper respiratory symptoms, lower respiratory symptoms and ears and hearing symptoms. Examination of the distribution of responses to items and the multitrait analyses enabled us to shorten the questionnaire by removing nine items which were redundant, not strongly endorsed or did not correlate strongly with its designated scale (see online supplementary table E1). The final QOL-PCD comprises 40 items on 10 scales, which were subjected to further psychometric analysis (see online supplementary table E2).

Analyses of the 40-item QOL-PCD confirmed that all items had strong correlations (≥0.63) with their intended scales. Few floor and ceiling effects were observed with the exception of: (1) floor effects on the social functioning scale (16.7% of respondents had low scores) and (2) ceiling effects were found on the physical functioning and ears and hearing scales, with 23.9% and 18.1% of respondents scoring the highest values, respectively.

Reliability: internal consistency and test–retest reliability

The QOL-PCD scales had moderate to strong internal consistency (0.74 to 0.94). Thirty-five participants repeated QOL-PCD after 10–14 days, providing evidence of stability across all scales with ICC ranging from 0.73 to 0.96 (table 2).

Table 2

Internal consistency of QOL-PCD scales measured by Cronbach's α and test–retest reliability measured by ICC

Validity

We had hypothesised that older age groups would report worse physical functioning, poorer vitality and more symptoms (table 3). Mean physical functioning scores were significantly worse in those who were older; average scores for the age group 18–22 years was 88.21 (SD=15.81), decreasing to 49.33 (SD=34.88) in the age group >55 years (p<0.001). Lower respiratory symptoms scores were highest in the youngest (18–22 years; M=58.76 (SD=16.72) and oldest age groups (>55 years (M=55.00 (SD=24.21)) and worst in the age group 37–55 years (M=36.30 SD=19.90). For patients with reported Pseudomonas aeruginosa, lower mean scores were reported for physical functioning and lower and upper respiratory symptoms; however, these did not reach statistical significance. For lung function, physical functioning scores were significantly lower in those with poor lung function, however, no significant associations were found on the upper and lower respiratory symptoms scores.

Table 3

QOL-PCD scales mean scores and SDs for participant characteristics where we had hypothesised an association a priori

As hypothesised, females reported significantly worse lower respiratory symptoms (p=0.004) (table 3). Although mean treatment burden scores were higher in males, this did not reach statistical significance (p=0.052). There was also no significant difference found between males and females on the social functioning scale (table 3).

Convergent validity was tested by examining correlations between scales measuring similar constructs on the QOL-PCD with other validated scales: SNOT-20 (upper airway), the SF-36 (generic health status) and the SGRQ-C (lower respiratory) (table 4). As expected, strong associations were found between the QOL-PCD upper respiratory symptoms and the SNOT-20 total score (r=0.60, p<0.01). Strong correlations were also found between QOL-PCD lower respiratory symptoms and SGRQ-C symptoms (r=0.69, p<0.001). On the SF-36, as hypothesised we found strong correlations between physical functioning and QOL-PCD physical scale (r=0.83, p<0.001), role-physical and QOL-PCD role functioning (r=0.83, p<0.001) and mental health with QOL-PCD emotional functioning scale (r=0.73, p<0.001). In contrast, as hypothesised, much weaker relationships were found between the QOL-PCD scores and generic questionnaires that measured dissimilar constructs (divergent validity). For example, the ear and hearing symptoms on the QOL-PCD correlated weakly with role functioning (r=0.39, p<0.001) on the SF-36 (table 4). Similarly, the QOL-PCD lower respiratory symptoms scale correlated more strongly than the upper respiratory scale, which focused on sinus symptoms, with SGRQ-C symptoms.

Table 4

Convergent validity testing: Spearman's rank correlation coefficients between scales from QOL-PCD and generic HRQOL measures (SNOT-20, SGRQ-C and SF-36)

Summary of distribution-based measures

The MCID was estimated from distribution analyses (table 5). Using 0.5 SD of baseline scores, MCID estimates ranged from 3.2 point (lower respiratory symptoms) to 15.2 points (physical functioning). Using SEM values derived from baseline scores, values ranged from 6.4 (lower respiratory symptoms) to 15.0 points (social functioning).

Table 5

Summary of distribution-based criteria to determine a minimum clinical significant change using 0.5 SD of the mean and SE of the measure (SEM) for each of the QOL-PCD scales

Discussion

We have shown that QOL-PCD (adult version) is a valid instrument to measure HRQOL in patients with PCD. Psychometric testing confirmed our measure to be robust, reliable and valid. QOL-PCD was developed in the UK and North America to ensure cross-cultural equivalence in English-speaking countries.19 The QOL-PCD has already been translated into Danish, Dutch, German (developed and linguistically validated for Germany and Switzerland), Greek and French; translations are progressing well in Turkish, Arabic, Spanish (European), Italian and Spanish (Latin America). This international approach is important for rare diseases since multinational clinical trials are needed to recruit sufficient patients.

Our study involved the recruitment of 72 patients with PCD from centres in North America (n=38) and the UK (n=34). All age groups were represented in the study, and participants had a range of disease severity (FEV1% predicted: 26%–115%). This collaborative effort allowed us to recruit sufficient participants with this rare disease. However, given the modest size of the study population, we conducted a multitrait analysis to develop the scales rather than an exploratory factor analysis (which would require >200 patients), or more complex analysis such as Rasche analysis (which would require >100 patients).33 The multitrait analysis supported the conceptual foundations of 10 scales with 40 items and has been recommended for use in the development of HRQOL measures, which are expected to have correlations across domains (eg, increased respiratory symptoms are likely to lead to decreased physical functioning). QOL-PCD had excellent item-to-total correlations and strong internal consistency across all scales (Cronbach's α 0.74 to 0.94). More complex analysis including factor analysis and Rasche analysis may be an option in future studies if we are able to recruit larger sample sizes.

All scales were stable in an analysis of test–retest reliability over 10–14 days. QOL-PCD correlated with generic HRQOL measures (SF-36, SGRQ-C, SNOT-20). As expected, there were stronger relationships between scales assessing similar rather than dissimilar constructs. As hypothesised, physical functioning scores were highest for those with FEV1>75%. However, FEV1 did not correlate with lower respiratory symptoms scales. FEV1 may be a poor measure for disease severity in patients with PCD as demonstrated in studies showing normal spirometry in individuals with PCD with substantial structural lung changes on chest CT.3 We recommend investigation into the association between QOL-PCD and other outcome measures assessing disease severity in PCD, such as chest CT,3 ,7 MRI8 and lung clearance index9–12 in the future.

Worsening QOL with age has previously been described in a PCD study using generic measures34 and decreases with age were also found in this study for both physical functioning and lower respiratory symptoms. Interestingly, the lowest QOL-PCD score in the upper and lower respiratory symptom scales were in our age group 37–55 years, with scores increasing again in the age group >55 years. Furthermore, the age group >55 years had significantly lower FEV1 compared with the younger age group combined (p=0.019). This may reflect specific challenges for this younger age group in comparison to those who are older, for example, lack of time to fit in treatments due to careers and families, or perhaps a survivor effect that is, those who have survived to 55 years may have milder symptoms. This highlights the kind of nuanced information that can be derived from HRQOL measures, reflecting the impact of disease management (ie, adherence to prescribed treatments) and progression of disease. Caution should be exercised in the interpretation of these findings where there is a limited sample size in each age group.

By facilitating discussion on issues that are of importance to patients, clinicians can focus attention on their patient's perceptions of their illness, facilitating collaborative care and shared medical decision-making. Moreover, patient's perceptions of improved functioning that are not reflected in other physiological outcomes may be important factors in promoting adherence to treatments.

QOL-PCD has also been developed for children (aged 6–12 years), adolescents (aged 13–17 years) and for parents (proxy) of children aged 6–12 years. QOL-PCD has been translated into six conceptually and linguistically equivalent language versions; a number of further translations are in progress, each following a protocol-led process of forward and back translation followed by cognitive testing. Validation of the remaining age-specific tools and different languages is underway.

We had planned to measure responsiveness in this study. Participants who contacted the study team towards the start of a respiratory exacerbation were asked to complete the questionnaire during a respiratory exacerbation. However, only 10 participants reported an exacerbation within the relatively short study period, and time periods from the well to the ill states varied considerable between patients (median of 54 days, range 3–152 days). Despite the small numbers of patients who completed QOL-PCD during an exacerbation (n=10), most scales evidenced worse mean scores in comparison to these patients' stable state. Scores for three subscales did not change during exacerbations (ears and hearing symptoms, social and emotional functioning) (figure 1) perhaps suggesting that chest exacerbations have minimal effects on these domains of functioning. Statistically significant results were also found (see online supplementary table E3); however, these were difficult to interpret given the small numbers. QOL-PCD is now being used in a randomised controlled study of azithromycin prophylaxis in PCD;35 we aim to assess the responsiveness to treatment in this study.

Figure 1

Difference in quality of life-primary ciliary dyskinesia between a stable baseline and a day during an exacerbation. (A) Lower respiratory symptoms scores. (B) Physical functioning (n=10).

The adult version of QOL-PCD is ready for use in clinical trials to assess the benefits of medications or non-pharmacological interventions. It can also be used to understand the natural course and progression of the disease in terms of its effects on physical, emotional, role and social functioning. We also propose QOL-PCD as a tool to be used at annual assessments, providing a broad assessment of well-being, as perceived by the patient.

Before QOL-PCD can effectively be used to assess intervention studies, it is important to determine the MCID score to allow for interpretation of changes in scores. We conducted preliminary analysis to estimate the MCID through distribution methods. This method is considered only supportive of an MCID with an intervention required to produce an accurate determination of change. QOL-PCD is currently being included in the first randomised clinical trial for PCD. This will facilitate the MCID for each scale to be determined.

In summary, we have developed19 and validated the first HRQOL instrument specific for PCD. QOL-PCD is valid and reliable; it is short and easy for patients to complete and provides a promising outcome measure for use in clinical trials and clinical practice.

Acknowledgments

We appreciate the ongoing collaboration with other Site Principal Investigators and Lead Study Coordinators in the Genetic Disorders of Mucociliary Clearance who recruited adults with primary ciliary dyskinesia (PCD) for this study, including Michael R Knowles, MD and Kelli M Sullivan at the University of North Carolina in Chapel Hill, NC; Thomas W Ferkol, MD and Jane Quante at Washington University School of Medicine, St Louis, MO; and Melody Miki at The Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada. We are grateful to the participants in this study. Members of the PCD Foundation, North America (Director, Michele Manion) and PCD Support Group, UK (Chair, Fiona Copeland) contributed to all aspects of study conduct.

References

View Abstract

Footnotes

  • Contributors MWL, SDD, ALQ and JSL had the concept for this study. All authors contributed to the study design. JSL and LB managed the conduct of the study and analysed the data. LB, MWL, SDD, ALQ and JSL discussed the analysed data and agreed the resultant 40-item QOL-PCD. LB drafted the manuscript. All authors contributed to iterations and approved the final document. JSL is accountable for the accuracy and integrity of the data.

  • Funding This research was funded by grant support to JSL, ALQ and MWL by funding from the European Union's Seventh Framework Programme under EC-GA No. 305404 BESTCILIA; by grant support to SDD and MWL: U54HL096458 from the National Institutes of Health (NIH) through the Genetic Disorders of Mucociliary Clearance Consortium, an initiative of the NIH Office of Rare Diseases Research at the National Center for Advancing Translational Science and the National Heart, Lung and Blood Institute; by grant support to SDD by Maya's March, The Hospital for Sick Children Foundation, Toronto, Ontario, Canada; ALQ was supported from an investigator-initiated grant, Gilead Sciences; The National PCD Centre in Southampton is commissioned and funded by NHS England. Research in Southampton is supported by NIHR Southampton Respiratory Biomedical Research Unit, NIHR Wellcome Trust Clinical Research Facility and The AAIR Charity (Reg. No. 1129698). JSL, MWL and LB are members of ERS Task Force for PCD Diagnostics (ERS TF-2014-04) and EU-funded COST Action BEAT-PCD (BM1407).

  • Competing interests None declared.

  • Patient consent Obtained.

  • Ethics approval The study was approved by the National Research Ethics Service, UK (UK 07/Q1702/109), the Research Ethics Board at the Hospital for Sick Children in Toronto, Canada and the Institutional Review Boards at the University of North Carolina, Chapel Hill.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles

  • Airwaves
    The Triumvirate