METHODS

Patients

Consecutive adult patients in whom OSA was clinically suspected and whose polysomnographic recording was abnormal were eligible for the study. OSA was diagnosed in patients with an apnoea-hypopnoea index (AHI) of ≥15 (an apnoeic event being defined as a cessation of the nasal flow for at least 10 seconds and hypopnoea as a 50% decrease in the nasal flow signal associated with a desaturation of more than 3%¹² and/or arousal). After the initial evaluation, therapeutic decisions were left to the patient and the treating physician. In those who selected nasal CPAP, its use was monitored and patients were asked to estimate the number of hours of CPAP used per night.

Sleep Apnoea Quality of Life Index (SAQLI)

The SAQLI is a 35 item OSA specific questionnaire composed of four core domains: daily functioning, social interactions, emotional functioning, and symptoms. It was developed to record key elements of the disease that are important to patients and for use as an evaluative instrument—that is, as a clinical outcome in clinical trials. The “symptoms” domain of the SAQLI is individualised: patients are asked to select from a list of 21 items the five most important symptoms they have experienced during the previous 4 weeks. Thus, a wide spectrum of symptoms may be chosen by patients so that each respondent will answer a different set of questions.

The SAQLI was translated from English to French following existing guidelines to preserve equivalence.¹³ This process involved (1) two translations of the questionnaire from English to French; (2) back translation from French to English; (3) comparison of the source and final versions; and (4) pre-testing of the French version on five patients with sleep apnoea.

Study design

Once the diagnosis of OSA had been made and before the initiation of OSA specific treatment, the SAQLI was administered to 50 consecutive patients. At the same time, the patients filled in four other questionnaires measuring constructs related to those measured by the SAQLI.

Medical Outcome Survey - Short Form 36 (SF-36) ^9,14

The SF-36 is a generic self-completed questionnaire that measures eight dimensions of health: physical functioning, role limitation due to physical problems, role limitation due to emotional problems, social functioning, mental health, energy/vitality, bodily pain, and general health perceptions.

Symptom Checklist - 90 (SCL-90) ¹⁵

The SCL-90 contains 90 items relating to nine different domains: anxiety, depression, hostility, obsessive-compulsiveness, sensitivity, sleeping disturbances, agoraphobia, somatisation, and psychoticism. We limited our use of the SCL-90 to the depression and hostility domains.

Beck Depression Inventory (BDI) ¹⁶

The BDI is a 21 item commonly used traditional instrument that was developed specifically to identify depression. It also has been extensively used as an evaluative instrument to monitor response to treatment in clinical trials.

Epworth Sleepiness Scale (ESS) ¹⁷

The ESS is a simple self-administered eight item questionnaire which measures the probability of falling asleep in eight specific situations that are commonly met in daily life.

At their 3 month follow up visit the patients again completed the same set of questionnaires whether or not they had received any treatment for OSA during this period. The respondents were then unaware of their previous responses. In addition, patients were asked to make a global rating of changes in their OSA related symptoms, daily life activities, social interactions, and emotions over the study period. For instance, they were asked: “Overall, has there been any change in your social life since the last time you saw us?”. Changes were scored on a 15 point scale ranging from –7 (a very great deal worse) to 0 (no change) to +7 (a very great deal better). Administration of these five questionnaires was supervised and took, on average, 45 minutes.

Statistics

Baseline and sample size

Descriptive statistics (proportions, means and standard deviations) were used to describe the study population at baseline. We calculated that 45 patients were needed if moderate (r=0.50) but statistically significant correlations were to be detected in the baseline discriminative analyses at the 0.01 level (β error 0.15).¹⁸

The SAQLI was analysed directly from the scores recorded. Individual items were equally weighted. The results were expressed as the mean score per item (ranging from 1 to 7) within each domain. The other questionnaires were analysed as advocated by their respective authors.

Reliability and internal consistency

Test-retest reliability was determined by correlating the baseline and follow up results of patients who did not receive any treatment for OSA over the study period. It was calculated using intraclass correlation coefficient, an index that corrects correlation for systematic bias that may exist if all the patients score higher (or lower) after a period of observation.¹⁹ Internal consistency (the extent to which different items in an instrument are measuring the same construct) was determined for each domain using Cronbach's alpha statistics.²⁰ We purposely did not include in this analysis the “symptoms” domain of the SAQLI because its items are individualised, which predictably reduces the between patient variance in scores and, as a consequence, reliability and internal consistency of the domain.²¹

Discriminative properties

The extent to which the SAQLI can distinguish between groups of patients was measured.¹⁰ Cross sectional construct validity was evaluated by correlating baseline quality of life scores with physiological sleep data and other related measures. Throughout the regression analyses, given the multitude of comparisons involved, statistical significance was set at the 0.01 level. We also examined the ability of the questionnaire to distinguish between groups of patients (treated v untreated) in terms of change in quality of life during the study period using unpaired t tests.

Evaluative properties

The extent to which the SAQLI can capture changes in quality of life over time—that is, the responsiveness of the questionnaires—was examined.¹⁰ This was primarily tested as the ability of the questionnaires to detect statistically significant differences in scores in the patients who were treated over the study period using paired t tests. The standardised response mean comparing the magnitude of change with the standard deviation of change was computed.²² The larger the standardised response mean, the more responsive to change is the questionnaire. Longitudinal construct validity was then demonstrated by correlating within subject changes in quality of life scores over the study period with within subject changes in other quality of life indices, and by showing that correlations of changes in different measures conform with those expected if the SAQLI is measuring what it is supposed to.

Interpretability

For an evaluative instrument, a score is interpretable when it indicates whether a particular change in score represents a small, moderate, or large clinical improvement or deterioration.²³ The results of the global rating of change questions were compared with the changes in scores within domains. Those who scored –3, –2, –1, 1, 2, or 3 on the global rating of change question were classified as having experienced a “small change” in quality of life; those who scored –5, –4, 4, or 5 on the global rating of change question were classified as having experienced a “moderate change” in quality of life; and those who scored –7, –6, 6, or 7 were classified as having experienced a “large change” in quality of life. Within each category the mean absolute change in score in the OSA questionnaires was computed.

A priori predictions

The following a priori predictions regarding the direction and magnitude of the correlations were formulated. At baseline we anticipated no significant correlation between the quality of life scores and any of the physiological measures, and moderate to high correlations (0.4≤r≤0.7) between scores in each domain of the OSA questionnaires and the corresponding instruments. Also, given the expected inability of generic questionnaires to detect change over time, we anticipated weak to moderate correlations (0.2≤r<0.4) between changes in scores in the OSA questionnaires and changes in the corresponding instruments. The finding that the actual correlations meet these a priori predictions would strengthen inferences regarding the validity of the OSA specific questionnaires.¹⁰

RESULTS

Patients

Fifty consecutive patients agreed to participate in the questionnaire validation process; 47 were available at 3 month follow up. The demographic and clinical characteristics of the study population are summarised in table 1. Thirty three received treatment for OSA over the study period (31 nasal CPAP, one uvulopalatopharyngoplasty, and one oral appliance). Baseline clinical characteristics of the 33 patients who were treated were not different from those 14 who were not.

Reliability and internal consistency

Test-retest reliability was determined in the 14 patients who did not receive any treatment during the study period and was found to be excellent with intraclass correlation coefficients (r) ranging from 0.91 to 0.94. Cronbach's alpha was ≥0.90 (daily life activities 0.94; social interactions 0.90; emotions 0.93).

Discriminative properties

The observed cross sectional correlations supporting the discriminative validity of the questionnaires are shown in table 2. There was good correlation between the SAQLI and all other related measures. None of the correlations between physiological data measured at baseline and quality of life scores was significant. The magnitude of these correlations met our a priori predictions. Examination of the ability of the questionnaire to distinguish between groups of patients (treated v untreated) in terms of change in quality of life during the study period showed statistically significant differences in all the SAQLI domains (symptoms: difference in change scores (δ)=1.4 (95% CI 0.3 to 2.6), p=0.015; daily life activities: δ=1.1 (95% CI 0.3 to 1.9), p=0.0005; social interactions: δ=1.0 (95% CI 0.3 to 1.7), p=0.0004; emotions: δ=0.6 (95% CI 0.0 to 1.2), p=0.013). Most of the differences obtained in the SF-36 subscales were statistically significant (physical functioning, p=0.29; vitality, p=0.013; role - physical, p=0.044; social functioning, p=0.036; role - emotions, p=0.055). However, the p values in the SF-36 analysis were higher than those corresponding to the SAQLI domains, indicating less ability to distinguish between treated and untreated patients.

Evaluative properties

The ability of the SAQLI and the SF-36 to detect changes is summarised in table 3. The results are presented as within group differences in the treated group. The ability to detect change in the treated group was higher for the SAQLI than the SF-36. Longitudinal construct validity correlations are shown in table 4. There were moderate to high correlations between changes in the OSA questionnaire and the related instruments. As expected, these correlations were smaller than those obtained in the cross sectional analysis. Again, the magnitude of most of these correlations met our a priori predictions. Self-reported compliance with nasal CPAP was closely correlated with the measured compliance (r=0.74; p=0.004). However, we could not find any correlation between changes in quality of life scores and the number of hours per night on CPAP.

Interpretability

The comparison between the observed changes in scores and the categories created from the global rating of change are presented in table 5. For all domains of the SAQLI, the magnitude of the changes in scores increased as the patients perceived larger benefits from their treatment. Overall, a difference in score of 2 points represented a small change in the “symptoms” domain whereas a difference of 1 point indicated a small change in the three other domains of the questionnaire. These differences may be regarded as the “minimal clinically important differences” for the corresponding domains.

DISCUSSION

The development of any specific questionnaire should be preceded by the identification of the areas of quality of life most affected by the disease, an initial step described by Flemons et al³ which they conducted according to standard methodology.²⁴ We were initially concerned by the fact that the item generation and reduction phases of the SAQLI development included patients of whom 80% had attempted some type of treatment. We hypothesised that the inclusion of treated patients in a study aimed at the identification of the domains of quality of life most affected by the disease may affect the relevance of items selected for inclusion in the final questionnaire and its measurement properties. In this regard, the finding of our descriptive study of quality of life in French speaking patients with OSA² was reassuring as it resulted in the identification of items that were remarkably similar to those of the SAQLI. This represents a strong argument in support of the content validity—that is, the comprehensiveness of the instrument—and the cross cultural adaptability of the SAQLI.

The initial SAQLI testing by Flemons et al³ was limited to an examination of its discriminative properties in 24 patients before starting CPAP and its evaluative properties in 15 patients who completed at least a 4 week trial of CPAP. Correlations were measured between the SAQLI total score and the SF-36 subscales and total scores, as well as between the change in SAQLI total score and changes in the SF-36 subscales and total score. The SAQLI domains were not examined individually. None of these correlations was preceded by an a priori prediction regarding their direction and magnitude. The correlation between the SAQLI and SF-36 total scores was not statistically significant. Only one significant correlation was reported with the “vitality” domain of the SF-36. Higher correlations were found between the SAQLI rating of change in total score and in total SF-36 rating of change and five of its domains. In a larger study we found higher correlations between both the discriminative and evaluative properties. In addition, the magnitude of most of the correlations met our a priori predictions, which also further supports the construct validity of the results.

We predicted the lack of correlation between the quality of life scores and any of the physiological outcomes measured (such as the apnoea/hypopnoea index). Similar findings have emerged from the measurement of quality of life in a variety of respiratory diseases such as asthma²⁵ and chronic obstructive pulmonary disease.²⁶ The impact of OSA on quality of life is not limited to excessive daytime sleepiness and increased risk of cerebrovascular and cardiovascular events. OSA clearly affects important domains of quality of life which remain unexplored in the sleep laboratory. Physiological outcomes cannot be used as surrogate outcomes for quality of life in sleep apnoea. Quality of life should be measured directly.

The SAQLI was also responsive to changes in quality of life, as demonstrated by the statistically significant differences in score changes observed in those who received OSA specific treatment over a period of 3 months. The SAQLI was found to be more responsive to change than the SF-36. This has direct implications for sample size calculations in clinical trials. We conducted an analysis of the clinical significance of changes in score over time for each of the SAQLI domains and found that a 2 point difference in the “symptoms” domain corresponded to the difference at which average patients start rating themselves as either “a little better” or “a little worse”. This difference may be regarded as the “minimal clinically important difference”—that is, the smallest difference perceived by the average patient.²³ For the other domains of the questionnaire the “minimal clinically important difference” corresponded roughly to a score of 1 point, which is also the difference obtained by Flemons et al in their analysis of the minimal clinically important difference of the total score of the SAQLI.²⁷ We attributed this finding to the decision by Flemons et al to report the SAQLI scores as a total encompassing all its domains. While providing an overall assessment of quality of life, a single number may obscure important information from the individual domains. Improvements in one domain may accompany a deterioration in another, and changes in more than one domain will not be captured by an overall score. Accordingly, we recommend that SAQLI results should be presented as the mean score per item within each domain on a 7 point scale.

Another important finding of our study was the very high Cronbach's alpha attached to three of the questionnaire domains. Cronbach's alpha is a measure of internal consistency—that is, the extent to which different items in an instrument are measuring the same construct. High Cronbach's alpha values indicate that there is some redundancy in the items that form the domains. A shorter version of the SAQLI may retain its measurement properties.

Other considerations in selecting the SAQLI for clinical trials are that the questionnaire is interviewer administered and that its “symptoms” domain is individualised. Individualised questionnaires offer the potential of enhanced responsiveness when the instrument is used in clinical trials, while administration of the SAQLI by an interviewer ensures a high completion rate. However, these properties are at the expense of the questionnaire being rather sophisticated and time consuming. Also, in long term or large clinical trials, the ease and convenience of standardised items (as opposed to individualised items) may outweigh the benefits of individualised items.²⁸ Such considerations were important in the development of a standardised version of an asthma specific quality of life questionnaire that retained most of the measurement properties of its individualised counterpart.²⁸

We are often asked to comment on the issue of cross cultural adaptability, a problem that relates to the development, translation, or utilisation of questionnaires in languages other than English. We concur with Jones' commentary that “the purpose of translation is not to produce a literal conversion of the questionnaire but to convey the spirit of the questionnaire items into different languages and cultures”.²⁹ There is more and more evidence, including this study, that careful translation and back translation of quality of life questionnaires can produce non-English language versions that appear to behave in a very similar manner to their originals.

In conclusion, the SAQLI has strong content and construct validity and is responsive to changes in quality of life, but has to be administered by an interviewer. The internal structure of the questionnaire indicates some redundancy in items within its four domains. A shorter and self-administered version of the SAQLI would be a useful addition to outcomes measures in patient orientated research in OSA.