Article Text

PDF

Independent validation of the Sleep Apnoea Quality of Life Index
  1. Y Lacasse,
  2. C Godbout,
  3. F Sériés
  1. Centre de Recherche, Centre de Pneumologie, Hôf.pital Laval, Institut Universitaire de Cardiologie et de Pneumologie de l'Université Laval, Québec, Canada
  1. Correspondence to:
    Dr Y Lacasse, Centre de Pneumologie, Hôf.pital Laval, 2725 Chemin Ste-Foy, Ste-Foy, Québec G1V 4G5, Canada;
    Yves.Lacasse{at}med.ulaval.ca

Abstract

Background: Obstructive sleep apnoea (OSA) affects important domains of quality of life which remain unexplored by conventional sleep recordings. The objective of this study was to examine the measurement properties (both discriminative and evaluative) of the Sleep Apnoea Quality of Life Index (SAQLI), a new OSA specific quality of life questionnaire.

Methods: Consecutive patients recently diagnosed with OSA completed a French version of the SAQLI twice over a 3 month period. Its construct validity and responsiveness were tested by comparing baseline and change scores obtained in each domain (symptoms, activities, emotions, social interactions) with those of questionnaires measuring related constructs (SF-36, Epworth Scale, Beck Depression Inventory, and Symptom Checklist 90). The symptoms scores were also correlated with physiological measures obtained at baseline polysomnographic recording.

Results: Forty seven patients (40 men) of mean (SD) age 53 (10) years and mean (SD) apnoea/hypopnoea index 38 (21) participated in the study. During the study period 33 of the 47 patients were treated for OSA (31 with nasal CPAP, one with uvulopalatopharyngoplasty, and one with an oral appliance). Moderate to high correlations were found between the scores in each domain of the SAQLI and the corresponding instruments. There were significant differences in change scores between patients who were treated and those who were not, moderate correlations between SAQLI change scores and changes in the corresponding instruments, and no correlation between the symptoms scores and the baseline nocturnal features. Most of these correlations met the a priori predictions made regarding their magnitude.

Conclusion: The SAQLI has strong construct validity and is responsive to change in quality of life but has the disadvantage of having to be administered by an interviewer.

  • obstructive sleep apnoea
  • quality of life
  • questionnaire

Statistics from Altmetric.com

Prevalence surveys have estimated that 4% of middle aged men and 2% of middle aged women are affected by sleep apnoea.1 Obstructive sleep apnoea (OSA) has a significant impact on areas of patients' quality of life that are not only confined to excessive daytime sleepiness.2 In descriptive studies of health related quality of life in OSA,2,3 daily functioning, emotional function, and social interactions were impaired in most patients suffering from the disease.

In a recent systematic review of the effects of OSA and the effectiveness of continuous positive airways pressure (CPAP),4 Wright and collaborators consistently found small changes in daytime sleepiness measured in the laboratory. However, the authors concluded that improvements in quality of life indicators had not been adequately assessed. The measurement of physiological and laboratory parameters cannot be used as a surrogate outcome for quality of life in sleep apnoea, given the poor correlation between the impairment of quality of life and the severity of sleep apnoea.5 The impact of treatment on quality of life should be measured directly. In sleep related outcome research, the development and validation of disease specific health status measurement instruments were identified as most in need of improvement.6

In a small number of studies7,8 investigators have recently qualified and quantified the impact of OSA and its treatment on patients' quality of life using generic instruments such as the Medical Outcome Survey - Short Form 36.9 Although generic questionnaires are designed to measure all important aspects of quality of life, they are theoretically less likely to detect changes in quality of life than a disease specific questionnaire.10 Sleep specific questionnaires such as the Sleep Apnoea Quality of Life Index (SAQLI)3 and the Functional Outcomes of Sleep Questionnaire11 are emerging, but the measurement properties of these questionnaires by independent investigators remain to be determined.

The objectives of this study were (1) to examine the validity, reliability, responsiveness, and interpretability of the SAQLI, a new OSA specific quality of life questionnaire to be used in clinical trials, and (2) to determine whether this new questionnaire may become a useful addition to generic questionnaires (such as the SF-36) in the evaluation of quality of life of patients suffering from OSA in the setting of clinical trials.

METHODS

Patients

Consecutive adult patients in whom OSA was clinically suspected and whose polysomnographic recording was abnormal were eligible for the study. OSA was diagnosed in patients with an apnoea-hypopnoea index (AHI) of ≥15 (an apnoeic event being defined as a cessation of the nasal flow for at least 10 seconds and hypopnoea as a 50% decrease in the nasal flow signal associated with a desaturation of more than 3%12 and/or arousal). After the initial evaluation, therapeutic decisions were left to the patient and the treating physician. In those who selected nasal CPAP, its use was monitored and patients were asked to estimate the number of hours of CPAP used per night.

Sleep Apnoea Quality of Life Index (SAQLI)

The SAQLI is a 35 item OSA specific questionnaire composed of four core domains: daily functioning, social interactions, emotional functioning, and symptoms. It was developed to record key elements of the disease that are important to patients and for use as an evaluative instrument—that is, as a clinical outcome in clinical trials. The “symptoms” domain of the SAQLI is individualised: patients are asked to select from a list of 21 items the five most important symptoms they have experienced during the previous 4 weeks. Thus, a wide spectrum of symptoms may be chosen by patients so that each respondent will answer a different set of questions.

The SAQLI was translated from English to French following existing guidelines to preserve equivalence.13 This process involved (1) two translations of the questionnaire from English to French; (2) back translation from French to English; (3) comparison of the source and final versions; and (4) pre-testing of the French version on five patients with sleep apnoea.

Study design

Once the diagnosis of OSA had been made and before the initiation of OSA specific treatment, the SAQLI was administered to 50 consecutive patients. At the same time, the patients filled in four other questionnaires measuring constructs related to those measured by the SAQLI.

Medical Outcome Survey - Short Form 36 (SF-36) 9,14

The SF-36 is a generic self-completed questionnaire that measures eight dimensions of health: physical functioning, role limitation due to physical problems, role limitation due to emotional problems, social functioning, mental health, energy/vitality, bodily pain, and general health perceptions.

Symptom Checklist - 90 (SCL-90) 15

The SCL-90 contains 90 items relating to nine different domains: anxiety, depression, hostility, obsessive-compulsiveness, sensitivity, sleeping disturbances, agoraphobia, somatisation, and psychoticism. We limited our use of the SCL-90 to the depression and hostility domains.

Beck Depression Inventory (BDI) 16

The BDI is a 21 item commonly used traditional instrument that was developed specifically to identify depression. It also has been extensively used as an evaluative instrument to monitor response to treatment in clinical trials.

Epworth Sleepiness Scale (ESS) 17

The ESS is a simple self-administered eight item questionnaire which measures the probability of falling asleep in eight specific situations that are commonly met in daily life.

At their 3 month follow up visit the patients again completed the same set of questionnaires whether or not they had received any treatment for OSA during this period. The respondents were then unaware of their previous responses. In addition, patients were asked to make a global rating of changes in their OSA related symptoms, daily life activities, social interactions, and emotions over the study period. For instance, they were asked: “Overall, has there been any change in your social life since the last time you saw us?”. Changes were scored on a 15 point scale ranging from –7 (a very great deal worse) to 0 (no change) to +7 (a very great deal better). Administration of these five questionnaires was supervised and took, on average, 45 minutes.

Statistics

Baseline and sample size

Descriptive statistics (proportions, means and standard deviations) were used to describe the study population at baseline. We calculated that 45 patients were needed if moderate (r=0.50) but statistically significant correlations were to be detected in the baseline discriminative analyses at the 0.01 level (β error 0.15).18

The SAQLI was analysed directly from the scores recorded. Individual items were equally weighted. The results were expressed as the mean score per item (ranging from 1 to 7) within each domain. The other questionnaires were analysed as advocated by their respective authors.

Reliability and internal consistency

Test-retest reliability was determined by correlating the baseline and follow up results of patients who did not receive any treatment for OSA over the study period. It was calculated using intraclass correlation coefficient, an index that corrects correlation for systematic bias that may exist if all the patients score higher (or lower) after a period of observation.19 Internal consistency (the extent to which different items in an instrument are measuring the same construct) was determined for each domain using Cronbach's alpha statistics.20 We purposely did not include in this analysis the “symptoms” domain of the SAQLI because its items are individualised, which predictably reduces the between patient variance in scores and, as a consequence, reliability and internal consistency of the domain.21

Discriminative properties

The extent to which the SAQLI can distinguish between groups of patients was measured.10 Cross sectional construct validity was evaluated by correlating baseline quality of life scores with physiological sleep data and other related measures. Throughout the regression analyses, given the multitude of comparisons involved, statistical significance was set at the 0.01 level. We also examined the ability of the questionnaire to distinguish between groups of patients (treated v untreated) in terms of change in quality of life during the study period using unpaired t tests.

Evaluative properties

The extent to which the SAQLI can capture changes in quality of life over time—that is, the responsiveness of the questionnaires—was examined.10 This was primarily tested as the ability of the questionnaires to detect statistically significant differences in scores in the patients who were treated over the study period using paired t tests. The standardised response mean comparing the magnitude of change with the standard deviation of change was computed.22 The larger the standardised response mean, the more responsive to change is the questionnaire. Longitudinal construct validity was then demonstrated by correlating within subject changes in quality of life scores over the study period with within subject changes in other quality of life indices, and by showing that correlations of changes in different measures conform with those expected if the SAQLI is measuring what it is supposed to.

Interpretability

For an evaluative instrument, a score is interpretable when it indicates whether a particular change in score represents a small, moderate, or large clinical improvement or deterioration.23 The results of the global rating of change questions were compared with the changes in scores within domains. Those who scored –3, –2, –1, 1, 2, or 3 on the global rating of change question were classified as having experienced a “small change” in quality of life; those who scored –5, –4, 4, or 5 on the global rating of change question were classified as having experienced a “moderate change” in quality of life; and those who scored –7, –6, 6, or 7 were classified as having experienced a “large change” in quality of life. Within each category the mean absolute change in score in the OSA questionnaires was computed.

A priori predictions

The following a priori predictions regarding the direction and magnitude of the correlations were formulated. At baseline we anticipated no significant correlation between the quality of life scores and any of the physiological measures, and moderate to high correlations (0.4≤r≤0.7) between scores in each domain of the OSA questionnaires and the corresponding instruments. Also, given the expected inability of generic questionnaires to detect change over time, we anticipated weak to moderate correlations (0.2≤r<0.4) between changes in scores in the OSA questionnaires and changes in the corresponding instruments. The finding that the actual correlations meet these a priori predictions would strengthen inferences regarding the validity of the OSA specific questionnaires.10

RESULTS

Patients

Fifty consecutive patients agreed to participate in the questionnaire validation process; 47 were available at 3 month follow up. The demographic and clinical characteristics of the study population are summarised in table 1. Thirty three received treatment for OSA over the study period (31 nasal CPAP, one uvulopalatopharyngoplasty, and one oral appliance). Baseline clinical characteristics of the 33 patients who were treated were not different from those 14 who were not.

Table 1

Clinical characteristics of the study population (n=47)

Reliability and internal consistency

Test-retest reliability was determined in the 14 patients who did not receive any treatment during the study period and was found to be excellent with intraclass correlation coefficients (r) ranging from 0.91 to 0.94. Cronbach's alpha was ≥0.90 (daily life activities 0.94; social interactions 0.90; emotions 0.93).

Discriminative properties

The observed cross sectional correlations supporting the discriminative validity of the questionnaires are shown in table 2. There was good correlation between the SAQLI and all other related measures. None of the correlations between physiological data measured at baseline and quality of life scores was significant. The magnitude of these correlations met our a priori predictions. Examination of the ability of the questionnaire to distinguish between groups of patients (treated v untreated) in terms of change in quality of life during the study period showed statistically significant differences in all the SAQLI domains (symptoms: difference in change scores (δ)=1.4 (95% CI 0.3 to 2.6), p=0.015; daily life activities: δ=1.1 (95% CI 0.3 to 1.9), p=0.0005; social interactions: δ=1.0 (95% CI 0.3 to 1.7), p=0.0004; emotions: δ=0.6 (95% CI 0.0 to 1.2), p=0.013). Most of the differences obtained in the SF-36 subscales were statistically significant (physical functioning, p=0.29; vitality, p=0.013; role - physical, p=0.044; social functioning, p=0.036; role - emotions, p=0.055). However, the p values in the SF-36 analysis were higher than those corresponding to the SAQLI domains, indicating less ability to distinguish between treated and untreated patients.

Table 2

Correlations between the SAQLI and related questionnaires

Evaluative properties

The ability of the SAQLI and the SF-36 to detect changes is summarised in table 3. The results are presented as within group differences in the treated group. The ability to detect change in the treated group was higher for the SAQLI than the SF-36. Longitudinal construct validity correlations are shown in table 4. There were moderate to high correlations between changes in the OSA questionnaire and the related instruments. As expected, these correlations were smaller than those obtained in the cross sectional analysis. Again, the magnitude of most of these correlations met our a priori predictions. Self-reported compliance with nasal CPAP was closely correlated with the measured compliance (r=0.74; p=0.004). However, we could not find any correlation between changes in quality of life scores and the number of hours per night on CPAP.

Table 3

Rating of change in the SAQLI and SF-36 questionnaires in the treated group (n=33)

Table 4

Correlations in ratings of change between the SAQLI and related questionnaires

Interpretability

The comparison between the observed changes in scores and the categories created from the global rating of change are presented in table 5. For all domains of the SAQLI, the magnitude of the changes in scores increased as the patients perceived larger benefits from their treatment. Overall, a difference in score of 2 points represented a small change in the “symptoms” domain whereas a difference of 1 point indicated a small change in the three other domains of the questionnaire. These differences may be regarded as the “minimal clinically important differences” for the corresponding domains.

Table 5

Differences in scores according to the magnitude of perceived change

DISCUSSION

The development of any specific questionnaire should be preceded by the identification of the areas of quality of life most affected by the disease, an initial step described by Flemons et al3 which they conducted according to standard methodology.24 We were initially concerned by the fact that the item generation and reduction phases of the SAQLI development included patients of whom 80% had attempted some type of treatment. We hypothesised that the inclusion of treated patients in a study aimed at the identification of the domains of quality of life most affected by the disease may affect the relevance of items selected for inclusion in the final questionnaire and its measurement properties. In this regard, the finding of our descriptive study of quality of life in French speaking patients with OSA2 was reassuring as it resulted in the identification of items that were remarkably similar to those of the SAQLI. This represents a strong argument in support of the content validity—that is, the comprehensiveness of the instrument—and the cross cultural adaptability of the SAQLI.

The initial SAQLI testing by Flemons et al3 was limited to an examination of its discriminative properties in 24 patients before starting CPAP and its evaluative properties in 15 patients who completed at least a 4 week trial of CPAP. Correlations were measured between the SAQLI total score and the SF-36 subscales and total scores, as well as between the change in SAQLI total score and changes in the SF-36 subscales and total score. The SAQLI domains were not examined individually. None of these correlations was preceded by an a priori prediction regarding their direction and magnitude. The correlation between the SAQLI and SF-36 total scores was not statistically significant. Only one significant correlation was reported with the “vitality” domain of the SF-36. Higher correlations were found between the SAQLI rating of change in total score and in total SF-36 rating of change and five of its domains. In a larger study we found higher correlations between both the discriminative and evaluative properties. In addition, the magnitude of most of the correlations met our a priori predictions, which also further supports the construct validity of the results.

We predicted the lack of correlation between the quality of life scores and any of the physiological outcomes measured (such as the apnoea/hypopnoea index). Similar findings have emerged from the measurement of quality of life in a variety of respiratory diseases such as asthma25 and chronic obstructive pulmonary disease.26 The impact of OSA on quality of life is not limited to excessive daytime sleepiness and increased risk of cerebrovascular and cardiovascular events. OSA clearly affects important domains of quality of life which remain unexplored in the sleep laboratory. Physiological outcomes cannot be used as surrogate outcomes for quality of life in sleep apnoea. Quality of life should be measured directly.

The SAQLI was also responsive to changes in quality of life, as demonstrated by the statistically significant differences in score changes observed in those who received OSA specific treatment over a period of 3 months. The SAQLI was found to be more responsive to change than the SF-36. This has direct implications for sample size calculations in clinical trials. We conducted an analysis of the clinical significance of changes in score over time for each of the SAQLI domains and found that a 2 point difference in the “symptoms” domain corresponded to the difference at which average patients start rating themselves as either “a little better” or “a little worse”. This difference may be regarded as the “minimal clinically important difference”—that is, the smallest difference perceived by the average patient.23 For the other domains of the questionnaire the “minimal clinically important difference” corresponded roughly to a score of 1 point, which is also the difference obtained by Flemons et al in their analysis of the minimal clinically important difference of the total score of the SAQLI.27 We attributed this finding to the decision by Flemons et al to report the SAQLI scores as a total encompassing all its domains. While providing an overall assessment of quality of life, a single number may obscure important information from the individual domains. Improvements in one domain may accompany a deterioration in another, and changes in more than one domain will not be captured by an overall score. Accordingly, we recommend that SAQLI results should be presented as the mean score per item within each domain on a 7 point scale.

Another important finding of our study was the very high Cronbach's alpha attached to three of the questionnaire domains. Cronbach's alpha is a measure of internal consistency—that is, the extent to which different items in an instrument are measuring the same construct. High Cronbach's alpha values indicate that there is some redundancy in the items that form the domains. A shorter version of the SAQLI may retain its measurement properties.

Other considerations in selecting the SAQLI for clinical trials are that the questionnaire is interviewer administered and that its “symptoms” domain is individualised. Individualised questionnaires offer the potential of enhanced responsiveness when the instrument is used in clinical trials, while administration of the SAQLI by an interviewer ensures a high completion rate. However, these properties are at the expense of the questionnaire being rather sophisticated and time consuming. Also, in long term or large clinical trials, the ease and convenience of standardised items (as opposed to individualised items) may outweigh the benefits of individualised items.28 Such considerations were important in the development of a standardised version of an asthma specific quality of life questionnaire that retained most of the measurement properties of its individualised counterpart.28

We are often asked to comment on the issue of cross cultural adaptability, a problem that relates to the development, translation, or utilisation of questionnaires in languages other than English. We concur with Jones' commentary that “the purpose of translation is not to produce a literal conversion of the questionnaire but to convey the spirit of the questionnaire items into different languages and cultures”.29 There is more and more evidence, including this study, that careful translation and back translation of quality of life questionnaires can produce non-English language versions that appear to behave in a very similar manner to their originals.

In conclusion, the SAQLI has strong content and construct validity and is responsive to changes in quality of life, but has to be administered by an interviewer. The internal structure of the questionnaire indicates some redundancy in items within its four domains. A shorter and self-administered version of the SAQLI would be a useful addition to outcomes measures in patient orientated research in OSA.

Call for peer reviewers

Clinical Evidence is a regularly updated evidence based journal available world wide both as a paper version and on the internet. Clinical Evidence urgently needs to recruit a number of new contributors. Contributors are health care professionals or epidemiologists with experience in evidence based medicine and the ability to write in a concise and structured way.

Clinical Evidence needs to recruit a number of new peer reviewers. Peer reviewers are health care professionals or epidemiologists with experience in evidence based medicine. As a peer reviewer you would be asked for your views on the clinical relevance, validity and accessibility of specific topics within the journal, and their usefulness to the intended audience (international generalists and health care professionals, possibly with limited statistical knowledge). Topics are usually 2000–3000 words in length and we would ask you to review between 2–5 topics per year. The peer review process takes place throughout the year, and our turnaround time for each review is ideally 10–14 days. If you are interested in becoming a peer reviewer for Clinical Evidence, please complete the peer review questionnaire at www.clinicalevidence.com or contact Polly Brown (pbrown{at}bmjgroup.com).

REFERENCES

View Abstract

Footnotes

  • Supported by the Quebec Lung Association. Yves Lacasse is a clinician scientist of the Fonds de la Recherche en Santé du Québec.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.