BACKGROUND Health related quality of life (HRQoL) measurement is important in determining the impact of disease on daily functioning and subsequently informing interventions. In cystic fibrosis (CF) generic HRQoL measures have been employed but these may not be sufficiently specific. The aim of the current work was to develop and validate a disease specific HRQoL measure for adults and adolescents with cystic fibrosis.
METHODS Areas of concern to adults and adolescents with CF were identified by unstructured interviews, self-administered questionnaires, consultation with multidisciplinary specialist staff, a review of the relevant literature, and examination of other HRQoL measures. Items for the questionnaire were generated on the basis of this process. Continued evaluation and development of the Cystic Fibrosis Quality of Life (CFQoL) questionnaire was undertaken by a process of statistical analysis and continued feedback from patients. The full testing and validation of the CFQoL questionnaire took place over four phases: (1) initial item generation and testing of a preliminary questionnaire, (2) testing and validation of the second version of the questionnaire, (3) test-retest reliability of a third and final version of the questionnaire, and (4) sensitivity testing of the final version of the questionnaire.
RESULTS Nine domains of functioning were identified using principal components analysis with varimax rotation. Internal reliability of the identified domains was demonstrated using Cronbach alpha coefficients (range 0.72–0.92) and item to total domain score correlations. Concurrent validity (ranger = 0.64–0.74), discriminatory ability between different levels of disease severity, sensitivity across transient changes in health (effect size range, moderate d = 0.56 to large d = 1.95), and test-retest reliability (r = 0.74–0.96) were also found to be robust.
CONCLUSIONS The CFQoL questionnaire is a fully validated disease specific measure consisting of 52 items across nine domains of functioning which have been identified by, and are of importance to, adolescents and adults with cystic fibrosis. This measure will be useful in clinical trials and longitudinal studies.
- health related quality of life
- cystic fibrosis
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Advances in the care and treatment of cystic fibrosis (CF) have resulted in most patients surviving into adulthood.1Despite these improvements, CF remains a progressive and ultimately fatal multisystem disease that has a heavy treatment regimen.2 Given this, a new goal for intervention in CF should be to measure and improve health related quality of life (HRQoL) in relation to medical and psychosocial interventions. The measurement of HRQoL complements clinical measures of disease status such as respiratory function tests. Once developed a questionnaire would be useful as (a) an outcome measure in clinical trials, (b) for the assessment of disease progression, and (c) for the monitoring of individual patients.
To date, HRQoL in adults with CF has been measured using either generic scales3-5 or disease specific respiratory measures.6-9 These measures were not developed for the CF population and are limited since they do not reflect areas of functioning that are particularly salient to the adult with CF. Because of this the data are likely to lack sensitivity and be problematic in their interpretation. This work aims to develop, test, and validate a disease specific measure of HRQoL for adults and adolescents with CF. The measure should include areas of functioning that are meaningful to adults with CF, be brief enough to be applied in a clinical setting (that is, completion time of about 10 minutes), be simple to administer and score (for use in clinical settings and postal surveys), and be sensitive enough to detect both changes in health within the individual and differences between levels of disease severity.
The development, testing, and validation of the Cystic Fibrosis Quality of Life (CFQoL) questionnaire took place over four phases: (1) initial item generation and testing of a preliminary questionnaire; (2) testing and validation of a second version of the questionnaire including concurrent and discriminative validity; (3) test-retest reliability of a third and final version of the questionnaire; and (4) sensitivity testing of the final version of the questionnaire. The step by step development and validation of the CFQoL is reflected in the structure of this paper with the various methods and results sections reported in sequential order for each stage of development and testing.
Phase 1: Initial item generation and testing of a preliminary questionnaire
Generation of an initial list of items likely to be important to adults with CF was informed by the following processes: (1) consideration of the general literature exploring HRQoL, (2) literature dealing with the psychosocial and physiological aspects of CF, (3) examination of existing generic and respiratory specific questionnaires, (4) the outcome of interviews with 60 CF adults conducted during the course of previous research10 which outlined areas of concern to them, (5) examination of the literature exploring HRQoL in CF, and (6) discussions with multidisciplinary staff from two specialist CF centres. These suggested several areas for inclusion in the questionnaire including physical functioning, social functioning, CF specific issues (chest symptoms, treatment, and body image), interpersonal relationships, career/college issues, and concerns for the future. Fifty items were constructed to represent these areas. Responses to the questionnaire were measured by a six point Likert scale ranging from 1 = very strongly agree to 6 = very strongly disagree. The response scale was arbitrary in a bid to encourage patients to suggest the most appropriate method of responding. A final screening item asked patients to report what they found most troublesome about their CF. This was included to ascertain whether any salient issues had been omitted. This initial questionnaire was designed as a framework from which items could be removed or added to in subsequent versions.
PATIENTS AND PROCEDURE
One hundred and five patients were recruited via the outpatient clinic of a specialist adult CF unit. Patients were randomly approached during routine visits and asked for consent to participate in the development of the questionnaire. The researcher had no prior knowledge of the patients or of their medical status. There was no ‘a priori’ exclusion of any patient attending the clinic on any grounds. To encourage genuine and meaningful responses no demographic or clinical details were taken for this initial and important phase of development.11 Patients self-completed the questionnaire which took approximately 10 minutes. Following completion, each patient was asked to comment on and criticise the structure and content of the measure. Responses to this were noted in order to include any refinements in a subsequent draft.
Of the 105 questionnaires completed, four were discarded because of incomplete data. Principal components analysis (PCA) with varimax rotation was conducted on items 1–50. PCA with varimax rotation is a statistical technique based on correlational principles which is applied to a single set of variables (questionnaire items) in order to highlight coherent subsets within the overall data set. In questionnaire design this technique is applied to identify the presence of domains (subscales reflecting different aspects of functioning) within a larger battery of items. The solution emerging from the PCA highlighted 10 principal factors with eigen values above 1.0 which accounted for 71.1% of the overall variance within the data set. An additional method of determining the relative importance of factors is the scree plot described by Cattell.12 This technique involves the graphical display of the descending variance accounted for by all the factors extracted by the analysis. The cut off point for this method is determined by the gradient of the slope. A scree plot confirmed the factors highlighted by the eigen values.
The factors identified were (1) physical functioning, mild to moderate impairment; (2) physical functioning, severe impairment; (3) social functioning; (4) treatment issues; (5) chest symptoms; (6) body image; (7) interpersonal relationships; (8) career concerns; (9) relationships at work; and (10) concerns for the future. The two physical functioning components were merged in order to create an overall domain of physical functioning which contained mild, moderate, and severe elements. The decision to do this was supported statistically by applying Cronbach alpha analysis to the merged physical functioning domain, the outcome of which indicated very good internal reliability (table 1). A single physical functioning domain is also clinically sensible and easily interpretable. With physical functioning merged into a single domain a total of nine domains of functioning remained.
Internal reliability of each domain was initially assessed using Cronbach alpha coefficients (table 1). Internal reliability is generally acceptable for factors with a Cronbach alpha coefficient of 0.7 or above.13 The exceptions to this were relationships at work, social functioning, and body image. These factors all contained items that were reworded slightly for subsequent versions, thereby eliminating ambiguities and leading to better reliability values. Because Cronbach alpha is influenced by the number of items in a domain, additional analyses using Pearson product moment correlation coefficients were conducted for all domains that contained only two items. Moderate to strong correlation coefficients emerged for treatment issues (r = 0.68), chest symptoms (r = 0.73), body image (r = 0.52), and relationships at work (r = 0.55). Overall, items loaded cohesively onto the emergent factors. However, there were several ambiguities and, in order to maintain the integrity of the domains that the factors represented, 10 items were omitted from the next draft of the questionnaire.
The acceptability and face validity of the questionnaire was confirmed by (a) the patient's willingness to complete it, (b) positive comments regarding its content, (c) indication by the patients that they would be happy to complete and comment on subsequent versions of the measure, (d) failure of the screening item to reveal additional issues, and (e) positive reaction by all members of the multidisciplinary CF team about the content and its relationship to the overall measurement aims. The feedback provided by both patients and all members of the CF team raised an additional 15 items for inclusion in the subsequent CFQoL. This included a new section covering emotional responses to CF. Patient feedback also led to the development of new response scales. For domains related to physical functioning, social functioning, chest symptoms, treatment issues and emotional responses, a six point Likert scale was applied with a range of 1 = all of the time to 6 = never, reflecting the fluctuating nature of CF. A two week time scale was applied across these domains. For the remaining domains a modification of the original scale was applied with 1 = strongly agree to 6 = strongly disagree.
Phase 2: Testing and validation of a second version of the CFQoL questionnaire
Thirty eight CF specific items were retained from the CFQoL1, either in their original form or in the reworded version, and the 15 new items were added to the second version of the questionnaire (CFQoL2). This generated a questionnaire with 53 items across nine domains of functioning.
PATIENTS AND PROCEDURE
The CFQoL2 questionnaire was completed by 223 adults with CF. The sample comprised 103 men and 120 women with a mean age of 25.2 years (range 14–52). Percentage of predicted forced expiratory volume in one second (FEV1) was also calculated (mean FEV1 = 55%, range 12–118%). The CFQoL2 questionnaire was distributed to coincide with attendance at the outpatient clinic. Patients self-completed the measure and returned completed questionnaires by post. For purposes of assessing the concurrent validity of the CFQoL2, the Short Form 36 item (SF-36) Health Status questionnaire was distributed in parallel. The SF-36 is a generic health status measure that has been well tested and validated across a number of populations.14-17 It is one of the briefest measures currently in use, but is still one of the most comprehensive.14
Testing the structure of the CFQoL2 questionnaire was conducted by applying PCA with varimax rotation, Cronbach alpha coefficients, and item to domain correlations. The PCA indicated a solution consisting of eight factors on the basis of eigen values of 1.0 and above and the examination of the data from a scree plot. These were (1) chest symptoms, physical functioning (mild to moderate impairment) and social functioning; (2) emotional functioning; (3) interpersonal relationships; (4) concerns for the future; (5) career concerns; (6) treatment issues; (7) physical functioning (severe impairment); and (8) body image.
For those factors with an eigen value below 2.0 (career concerns, treatment issues, and body image) an additional screening measure was used. The percentage of adults with CF who responded negatively across domains provided a measure of “importance to patients”. A negative response to domains is indicated by a raw score of 3 or less and is indicative of the difficulties patients are experiencing; 43% of patients responded negatively with regard to career concerns, 19% regarding treatment issues, and 29% concerning body image, so the retention of these domains was also warranted.
Physical functioning emerged as two factors that were again merged into a single domain. The rationale for this was that physical functioning is more easily interpreted and clinically sensible as a continuum of severity. Additionally, treating mild and moderate physical functioning separately from severe physical functioning may serve to generate floor and ceiling effects across each of these domains. For example, more severely affected individuals are likely to generate floor effects in the mild to moderate range whereas healthier individuals would generate ceiling effects across the severe range of items. Merging of the items serves to buffer this effect while still being sensitive to a range of disability across individuals. Statistical review supported this decision given that the Cronbach alpha coefficients (table 1) and the item to domain correlations (table 2) indicated that the domain was structurally robust. Item to domain correlations test whether items display a stronger correlation with the total domain score from their related domain compared with unrelated domains. To prevent overfitting of the item with related domain scores, the score from the item being considered was removed from the total domain score.
Also loading on to factor 1 were items that had previously emerged as separate factors (chest symptoms and social functioning). Treating these factors as a single domain may serve to confuse interpretability. Amalgamation of these items would lead to a loss of this specificity and the ability to respond in a clinical context. Cronbach alpha values (table 1) and item to domain correlations (table 2) demonstrated the integrity of the chest symptoms and social functioning domains as independent constructs. Because only two items were included in the chest symptoms domain a correlation analysis was conducted, the outcome of which revealed a strong relationship (r = 0.72).
On the basis of the statistical and clinical interpretation of the data, the following nine final domains of functioning were identified: physical functioning, social functioning, treatment issues, chest symptoms, emotional functioning, concerns for the future, interpersonal relationships, body image, and career issues. Both Cronbach alpha testing and item to domain correlation coefficients confirmed that the internal structure of the final nine domains were robust (tables 1 and2).
Method: patients and procedure
It is customary to assess the concurrent validity of a new questionnaire by comparing it with an established and well validated measure. The SF-36 questionnaire was completed by the 223 patients alongside the CFQoL2 questionnaire.
Correlations between the two questionnaires were only made across similar domains: physical functioning (r = 0.73), social functioning (r = 0.67), mental health/emotional functioning (r = 0.64), and total questionnaire scores (r = 0.74). All of the correlations were moderate to strong and very highly significant (p<0.001), indicating that the CFQoL2 questionnaire had good concurrent validity with the appropriate domains of the SF-36 questionnaire.
DISCRIMINATION OF THE CFQOL2 QUESTIONNAIRE
Method: patients and procedure
A method of assessing whether the measure can differentiate between patients with varying degrees of disease severity is to divide the population on the basis of percentage predicted FEV1.18 The overall group of 223 patients given the CFQoL2 questionnaire were divided into three disease severity groups: mild disease (FEV1 ⩾71% predicted), moderate disease (FEV1 41–70% predicted), and severe disease (FEV1 ⩽40% predicted). Domain comparisons between the groups were made using one way analysis of variance (ANOVA) with Tukey's honestly significant difference as a post hoc analysis for significant differences between groups.
Significant main effects were observed for all domains: physical functioning (F = 15.82, p<0.000), social functioning (F = 4.56, p<0.01), treatment issues (F = 4.44, p<0.01), chest symptoms (F = 11.68, p<0.000), emotional responses (F = 4.64, p<0.01), concerns for the future (F = 3.52, p<0.03), interpersonal relationships (F = 5.76, p<0.003), body image (F = 10.96, p<0.000), career concerns (F = 5.33, p<0.005), and total scores (F = 13.28, p<0.000). Post hoc analysis demonstrated that the CFQoL2 discriminated between levels of disease severity in the following ways: (1) between mild and moderate disease for the domains of body image, interpersonal relationships, concerns for the future, and total CFQoL scores; (2) between moderate and severe disease for physical functioning, social functioning, and chest symptoms; and (3) between mild and severe disease for all domains except concerns for the future.
To compare the discriminatory validity of the SF-36 a series of ANOVAs were also applied to comparative domains as with the assessment of concurrent validity. Significant main effects occurred for total SF-36 scores (F = 5.04, p<0.007) and for physical functioning (F = 17.43, p<0.000), with specific group differences between mild and severe groups for both domains and also between severe and moderate groups for the physical functioning domain. No group differences emerged for social functioning or mental health. These data indicated that the CFQoL had good discriminatory ability, particularly in comparison with the SF-36.
Phase 3: Test-retest reliability of the third version of the questionnaire
On the basis of the analyses described in phase 2, further minor amendments were made to the CFQoL resulting in a third and final version of the questionnaire consisting of 52 items across nine domains of functioning (appendix ). The test-retest procedure measured the stability of scores on the CFQoL over time. If the questionnaire is administered at time point 1 and then again 7–10 days later, the scores should remain relatively unchanged for patients who report no change in health status. This is important in determining whether changes measured by a questionnaire are genuine or an artefact of chance fluctuations.
PROCEDURE AND PATIENTS
To test the external reliability of the measure, the third version of the CFQoL questionnaire was posted to 200 adults with CF. A second questionnaire was sent to be completed within 7–10 days of the first. This time period was chosen to be long enough not to introduce memory confounds while being short enough to prevent fluctuations in disease status. Patients were also sent a further questionnaire alongside the second distribution of the CFQoL questionnaire which asked them to what extent they felt their health had changed since the first completion of the measure. Only patients who stated no change in health status and who completed both questionnaires within 7–10 days were included in the analysis.
One hundred patients completed and returned the questionnaire. Of these only 32 (17 men) fitted the inclusion criteria. The mean age of responders was 27 years (men 28 years, women 26 years). The overall age range was 16–53 years (men 18–40, women 16–53). Correlation analysis was applied to assess the degree of relationship between data completed at the two designated time points (table 3). The analysis indicated strong correlations (r >0.8) and explained 72.9–92.2% of overall variance between pairs (r 2 × 100). The only exception to this was for treatment issues (r = 0.74), explaining 55.2% of the variance. This indicated that the test-retest reliability for the CFQoL was robust.
Phase 4: Sensitivity testing of the third version of the CFQoL
Sensitivity testing examines whether a measure is able to detect either deterioration in health status or improvements brought about by an intervention. HRQoL using the third and final version of the CFQoL questionnaire was assessed in patients presenting with a pulmonary exacerbation. It was anticipated that improvements in pulmonary function as determined by percentage predicted FEV1, and possibly small changes in body mass index (BMI), would have occurred by the end of the treatment period.
PATIENTS AND PROCEDURE
Patients presenting at the clinic over a four month period who were diagnosed by the attending physician as presenting with an exacerbation of chest symptoms consented to participate in the study. Data were collected from 24 patients (nine men) of mean age 23 years (range 16–48). Participants completed the CFQoL questionnaire on diagnosis (time point 1) and again at the end of a two week period of intravenous antibiotic treatment (time point 2). Also collected at each time point were FEV1 and height and weight in order to calculate percentage predicted FEV1 and BMI.
Each domain was analysed separately by applying a series of related t tests and calculating the effect size of the change between time point 1 and time point 2. The analyses revealed a range of significant differences with large effect sizes for clinical status across the majority of domains (table 4). Higher values were attained at the second time point indicating an improvement in function.
Scoring and interpretation of scores
The scoring scales were designed to reflect positive and negative scoring. All items were scored 1 to 6, with the exception of item 6 (“Despite CF, over the last two weeks I have got around and done what I like”) which is reverse scored 6 to 1. Raw scores of 3 or under indicate a negative response. Transformed domain scores of 50 or less also indicate negative responses and suggest that the individual may be experiencing difficulties within that particular domain. Transformed scores translate to values between 0 and 100 and are achieved by following the equations outlined in appendix . A transformed score indicates the value that has been achieved out of a maximum of 100 with 100 indicating the most positive QoL levels possible.
The CFQoL questionnaire is a valid and reliable measure of HRQoL for adults and adolescents with CF. It is a patient derived measure which includes domains and response scales that have been determined by the patients and are therefore meaningful to them. Within each domain are sets of items, each of which tap into specific areas of HRQoL issues of importance to adults with CF. The outcome of this is that these can be assessed on a domain by domain basis or, more specifically, on an item by item basis if individual issues are to be evaluated in a therapeutic context. Response rates of 98% for face to face completion and 54% for postal means suggests a good compliance rate for the CFQoL.
Given that no attempt was made to eliminate sex biased questions from the measure, the CFQoL questionnaire will be able to identify disparities in HRQoL between men and women, should they exist. If items had been eliminated on the basis of sex differences important issues based on group membership may have been missed. It is important that sex is considered separately in health issues, as there is a wealth of literature that suggests that men and women respond very differently to poor health.19-23 Women generally report higher levels of physical disease, more pain, and more subjective or emotional symptoms than men. It is recommended that an initial analysis on the basis of sex should always be conducted, although where sex differences are not observed it is reasonable to proceed with a global analysis of the data.
The analysis of the CFQoL questionnaire has suggested that it may be appropriate for several different applications. Firstly, the data relating to the sensitivity of the CFQoL questionnaire have indicated that the measure is able to detect transient changes in health status. This is relevant for use in clinical trials (both medical or psychosocial in nature). Secondly, it is suitable for cross sectional comparison of groups given that it detected differences based on group membership which reflected varying levels of disease severity in adults with CF. Thirdly, the questionnaire would be useful in describing the longitudinal changes that take place in adults with CF as a function of progression and deterioration in disease status. The measure would be usefully applied at annual reviews, or even more frequently as the health of the patient begins to decline more rapidly. This would provide useful natural data for assessing the outcomes of various interventions including heart-lung transplantation, for which few baseline data have been collected in the existing studies,24-26 and also for end point stages in the terminal phase of the disease. Finally, the CFQoL questionnaire will also be useful on an individual basis to identify and intervene where problems are evidenced. This would provide a valuable adjunct to the diagnostic approaches already available, highlighting poor functioning on a wider level than merely physiological and serving to highlight areas that would be open to intervention on a more psychosocial level alongside the traditional medical interventions.
Final version of the CFQoL
SECTION 1: PHYSICAL FUNCTIONING
Please note that item 6 in this section is reverse scored. All other items throughout the questionnaire are scored 1 to 6 with the exception of this item which is scored 6 to 1.
Physical functioning has 10 items, therefore a score of between 0 and 100 will be derived by adding all 10 items (10/50 × 100).
SECTION 2: SOCIAL FUNCTIONING
Social functioning has 4 items therefore a score of between 0 and 100 will be derived by adding all 4 items (4/20 × 100).
SECTION 3: TREATMENT ISSUES AND CHEST SYMPTOMS
Treatments issues has 3 items therefore a score of between 0 and 100 will be derived by adding all 3 items (3/15 × 100).
Chest symptoms has 4 items therefore a score of between 0 and 100 will be derived by adding all 4 items (4/20 × 100).
SECTION 4: EMOTIONAL FUNCTIONING
Emotional functioning has 8 items therefore a score of between 0 and 100 will be derived by adding all 8 items (8/40 × 100).
SECTION 5: CONCERNS FOR THE FUTURE
Concerns for the future has 6 items therefore a score of between 0 and 100 will be derived by adding all 6 items (6/30 × 100).
SECTION 6: INTERPERSONAL RELATIONSHIPS
Interpersonal relationships has 10 items therefore a score of between 0 and 100 will be derived by adding all 10 items (10/50 × 100).
SECTION 7: BODY IMAGE
Body image has 3 items therefore a score of between 0 and 100 will be derived by adding all 3 items (3/15 × 100).
SECTION 8: CAREER CONCERNS
Career concerns has 4 items therefore a score of between 0 and 100 will be derived by adding all four items (4/20 × 100)