Article Text
PDF
Abstract
Background To address the growing burden of disease and long waiting lists for sleep services, a simplified two-stage model was developed and validated for identifying obstructive sleep apnoea (OSA) in primary care using a screening questionnaire followed by home sleep monitoring.
Methods 157 patients aged 25–70 years attending their primary care physician for any reason at six primary care clinics in rural and metropolitan regions of South Australia participated. The first 79 patients formed the development group and the next 78 patients the validation group. A screening questionnaire was developed from factors identified from sleep surveys, demographic and anthropometric data to be predictive of moderate to severe OSA. Receiver operating characteristic (ROC) curve analysis was used to validate the two-channel ApneaLink device against full polysomnography. The diagnostic accuracy of the overall two-stage model was then evaluated.
Results Snoring, waist circumference, witnessed apnoeas and age were predictive of OSA and incorporated into a screening questionnaire (ROC area under curve (AUC) 0.84, 95% CI 0.75 to 0.94, p<0.001). ApneaLink oximetry with a 3% dip rate was highly predictive of OSA (AUC 0.96, 95% CI 0.91 to 1.0, p<0.001). The two-stage diagnostic model showed a sensitivity of 0.97 (95% CI 0.81 to 1.00) and specificity of 0.87 (95% CI 0.74 to 0.95) in the development group, and a sensitivity of 0.88 (95% CI 0.60 to 0.98) and specificity of 0.82 (95% CI 0.70 to 0.90) in the validation group.
Conclusion A two-stage model of screening questionnaire followed by home oximetry can accurately identify patients with OSA in primary care and has the potential to expedite care for patients with this common sleep disorder.
- Obstructive sleep apnoea
- primary healthcare
- diagnosis
- questionnaires
- oximetry
- sleep apnoea
Statistics from Altmetric.com
Introduction
Obstructive sleep apnoea (OSA) affects 9–24% of middle-aged adults, with at least 4% of men and 2% of women suffering from symptomatic disease.1 These figures, however, probably underestimate the current prevalence of OSA given more recent population trends in obesity.2 3 OSA is associated with an increased risk of hypertension, motor vehicle accidents, neurocognitive impairment and cardiovascular disease.4–7 Treatment of OSA can reduce the health-related consequences of disease7 8 and is highly cost-effective.9 Despite increasing awareness of its adverse health consequences, OSA remains significantly underdiagnosed.10
A major impediment to OSA service access is the reliance on diagnostic laboratory-based polysomnography (PSG) which is labour-intensive, costly and has limited availability.11 Another impediment is the relative dearth of qualified sleep physicians. Simplified strategies have been proposed to address these issues, including clinical prediction models12–16 and home-based strategies incorporating portable sleep monitoring and autotitrating continuous positive airway pressure (CPAP).17 A recent study showed that primary responsibility for the care of OSA can be assumed by sleep-trained nurses, and therefore potentially other health professionals, without compromising patient outcomes.18
Significant potential exists to broaden the scope of OSA diagnosis and management within the primary care setting. Almost one-third of primary care patients have a high likelihood of OSA,19 yet doctors often fail to ask about features of OSA and patients frequently fail to report sleep-related symptoms.20 A possible barrier is the absence of a simple validated screening tool suitable for a busy primary care environment. The ideal diagnostic screen would contain no more than five items and be quick to administer and interpret without the need for specialised equipment or examination techniques.
The Berlin questionnaire is the only OSA questionnaire developed for and validated in primary care16 and categorises patients as either high or low risk for OSA based on self-reports of snoring, daytime sleepiness, hypertension and obesity. The 11 questions were chosen by a panel of sleep physicians without prior evaluation as to their respective discriminatory values. Although published a decade ago, the Berlin questionnaire is not widely used, possibly because of the time required for completion and scoring. Other screening tools have only been tested on selected populations in specialist sleep centres12–15 and, like the Berlin questionnaire, have complex scoring systems12–14 and/or require measurements of facial and oropharyngeal anatomy.15 Nevertheless, questionnaires alone may not provide a sufficient basis for diagnostic and treatment decisions in OSA. Consequently, suitably simple, accurate and validated strategies capturing both symptomatology and objective signs of overnight breathing disturbances are needed.
The aim of this study was to develop and validate a simplified two-stage method for identifying moderate to severe OSA in primary care consisting of an easy-to-administer screening questionnaire followed by home sleep apnoea monitoring.
Methods
A detailed account of the methodology is available in the online supplement. The study protocol was approved by the institutional research ethics committees at the Repatriation General Hospital and Flinders Medical Centre, South Australia, and participants provided written informed consent. The study was designed to meet the STARD guidelines for reports of diagnostic accuracy.21
Survey distribution and patient selection
Patients aged 25–70 years attending six primary care clinics for any reason between June 2007 and April 2008 were asked to complete a general health questionnaire, Epworth Sleepiness Scale (ESS)22 and Berlin questionnaire. Pregnant women and patients with significant cognitive impairment, a poorly-controlled psychiatric disorder or who had previously received treatment for OSA were excluded.
Reasoning that the true prevalence of moderate to severe OSA in our study population would be relatively low, an ‘OSA-enriched’ patient sample was selected for home sleep studies to minimise the CIs around estimates of diagnostic accuracy: four ‘high-risk’ patients to every one ‘low-risk’ patient (according to Berlin questionnaire results) underwent simultaneous home PSG and monitoring with a two-channel portable device. The first half of patients recruited formed the development group and the second half of patients formed the validation group.
Home sleep studies
Patients were visited at home by a trained sleep nurse who took anthropometric measurements and attached sleep recording devices. Full PSG was conducted using a Somte multichannel recorder (Compumedics, Melbourne, Australia). While home PSG is not considered the true gold standard for diagnosis of OSA, the Somte device has previously been validated against full PSG in the laboratory setting23 and, in a previous study of the Sleep Heart Health Study cohort, respiratory disturbance index measures from unattended home PSG were found to be comparable to laboratory-based PSG.24 A single experienced sleep technician performed manual scoring of all home PSGs according to internationally agreed criteria.25 An apnoea was defined as a cessation of nasal flow lasting ≥10 s. A hypopnoea was defined as a 50% decrease in nasal flow (or in both thoracic and abdominal excursions) lasting ≥10 s or a discernable decrease leading to a ≥3% oxygen desaturation or an EEG arousal. Moderate to severe OSA was defined as an apnoea-hypopnoea index (AHI) of ≥30/h. A detailed description of the Somte PSG device and scoring criteria are provided in the online supplement.
The ApneaLink device (ResMed, Sydney, Australia) is a portable two-channel monitor which records oxygen saturation and nasal flow. Data were automatically analysed using ApneaLink software Version 6.00 to derive: (1) an AHI20,50 from the airflow signal, with apnoea defined as a reduction in airflow to <20% of baseline and hypopnoea as a reduction in airflow to 20–50% of baseline for ≥10 s; and (2) an oxygen desaturation index with ≥3% oxygen desaturations (3%ODI). Patients who failed PSG and/or ApneaLink studies were asked to repeat simultaneous home monitoring.
Data analysis and statistics
Differences between baseline characteristics of patients in the development and validation groups were examined using independent sample Student t tests, Wilcoxon sum rank tests or χ2 tests where appropriate (Stata Version 11.0). Further details are provided in the online supplement.
Chi-square automatic interaction detection (CHAID)26 was used to identify variables predictive of an AHI ≥30/h in the development group and thus of potential use in the screening questionnaire (CHAID for Windows Version 6, Statistical Innovations Inc, 1993). A detailed description of CHAID and the choice of an AHI cut-off of 30/h to define moderate to severe OSA is provided in the online supplement. Potential predictor variables included gender, age, individual ESS and Berlin questionnaire items, total ESS score, body mass index (BMI), waist circumference and neck circumference. Variables predictive of OSA were entered into a logistic regression analysis and a simple scoring algorithm was derived using weightings from regression coefficients. Receiver operating characteristic (ROC) curve analyses were performed to assess the accuracy of the screening questionnaire as well as the ApneaLink-derived AHI20,50 and 3%ODI against a PSG AHI ≥30/h (Microsoft Office Excel 2003, Microsoft Corporation). The superior ApneaLink parameter was selected for inclusion in the second step of the diagnostic model. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive and negative likelihood ratios and overall test accuracy were calculated for both the questionnaire and ApneaLink to determine optimal cut-off values to maximise diagnostic efficiency. The accuracy of the final two-stage model for moderate to severe OSA was then evaluated in the validation group.
Results
The patient recruitment pathway is outlined in figure 1. Of the 1251 questionnaire packs returned from the primary care clinics, 461 patients (188 at high risk and 273 at low risk for OSA according to the Berlin questionnaire) were eligible for selection for home sleep studies. One hundred and seventy-six patients underwent home monitoring, with 157 performed successfully (123 patients at high risk and 34 at low risk for OSA). Nineteen patients were excluded owing to failed sleep studies, mainly from inadequate recording times or equipment failure, and either failed their second attempt or declined a repeat study. The development group consisted of the first 79 patients, while the remaining 78 patients made up the validation group.
Patient recruitment pathway. OSA, obstructive sleep apnoea; PSG, polysomnography.
Patient characteristics for the development and validation groups are shown in table 1. While both groups comprised predominantly middle-aged and overweight to obese individuals with approximately equal numbers of men and women, the development group was slightly older and had higher BMI, neck, waist and hip circumference measurements. The proportion of patients with moderate to severe OSA was higher in the development group. There was no difference in Berlin questionnaire risk rating or ESS score.
Characteristics of patients in development and validation groups
Development data set
Screening questionnaire development
Four variables were significantly predictive of moderate to severe OSA: (1) Berlin Questionnaire item 4: ‘Has your snoring ever bothered other people?’; (2) waist circumference (men >102 cm, women >88 cm); (3) age ≥50 years; (4) Berlin questionnaire item 5: ‘Has anyone noticed that you stop breathing during your sleep?’. The results of the logistic regression analysis are shown in table 2. A 4-item screening tool was created and named the ‘OSA50’ questionnaire (figure 2). Points are allocated to each question, with snoring and waist circumference having the highest score of 3 points each and age and witnessed apnoeas gaining 2 points each, to give a maximum total score of 10 points.
Logistic regression analysis: factors associated with an apnoea-hypopnoea index ≥30/h as determined by χ2 automatic interaction detection analysis
OSA50 screening questionnaire.
On ROC curve analysis, the OSA50 questionnaire was significantly predictive of moderate to severe OSA with an AUC of 0.84 (95% CI 0.75 to 0.94, p<0.001, figure 3). Using a cut-off score of ≥5/10, the screening questionnaire had a sensitivity of 100% (95% CI 86% to 100%), NPV of 100% (95% CI 73% to 100%), specificity of 29% (95% CI 17% to 44%) and PPV of 48% (95% CI 35% to 63%).
Receiver operating characteristic curve showing the performance of the OSA50 screening questionnaire in discriminating patients with moderate to severe obstructive sleep apnoea (apnoea-hypopnoea index ≥30/h) in the development group (n=79). AUC, area under the curve.
Validation of Apnealink monitor
Two patients with successful home PSGs included in the questionnaire development had failed ApneaLink studies and were excluded from further analysis, leaving 77 patients for validation of the ApneaLink monitor and analysis of the overall two-stage diagnostic model. ROC curves for the ApneaLink 3%ODI and AHI20,50 against PSG in the development group are shown in figure 4. Both the 3%ODI and AHI20,50 were highly predictive of moderate to severe OSA with ROC AUC values of 0.96 (95% CI 0.91 to 1.00, p<0.001) and 0.95 (95% CI 0.89 to 1.0, p<0.001), respectively. The 3%ODI was selected for use in the two-stage model because oximetry was technically more reliable than nasal airflow measurements. Sixteen (9%) of the 176 initial home sleep studies failed due to an inadequate airflow signal compared with only five (3%) with failed oximetry.
Receiver operating characteristic curves showing the performance of the ApneaLink 3%ODI and AHI20,50 in diagnosing moderate to severe obstructive sleep apnoea (AHI ≥30/h) in the development group. AHI, apnoea-hypopnoea index; ODI, oxygen desaturation index.
Two-stage diagnostic model
The diagnostic characteristics of the two-stage model are shown in tables 3A and 4. Thirty patients in the development group had an AHI of ≥30/h. Using cut-off values of ≥5/10 for the OSA50 questionnaire and ≥16/h for the 3%ODI, the two-stage model was capable of identifying moderate to severe OSA with a high sensitivity and specificity and had an overall diagnostic accuracy (sum of the true positive and true negative rate) of 91%.
Contingency tables for development and validation groups: (A) Development group (n=77). (B) Validation group (n=78)
Accuracy of two-stage diagnostic model (OSA50 score ≥5 and ApneaLink 3%ODI ≥16/h) for identifying moderate to severe OSA
Validation data set
The two-stage model was prospectively applied to the validation sample (tables 3B and 4). Sixteen patients (21%) in the validation group were confirmed to have moderate to severe OSA. The performance of the OSA50 questionnaire in the validation group was similar to that in the development group, with a ROC AUC of 0.75 (95% CI 0.59 to 0.90, p<0.001). Twenty of the 78 patients (26%) in the validation sample would have been excluded from further testing on the basis of a negative OSA50 questionnaire, only one of whom was positive for moderate to severe OSA on PSG. The two-stage model correctly identified moderate to severe OSA with a sensitivity and specificity of >80%, a very high NPV and overall diagnostic accuracy of 83%.
Discussion
Our results demonstrate that a simplified diagnostic model consisting of a screening questionnaire followed by home oximetry can identify patients in primary care with moderate to severe OSA with an overall accuracy exceeding 80%. We targeted moderate to severe OSA as more severely affected patients have the highest morbidity and also adhere and respond to treatment better than those with mild disease.27 Compared with the standard pathway of care involving specialist review and laboratory-based testing, our diagnostic model has the potential to offer a low-cost and easily accessible strategy which can be initiated in primary care.
Our clinical prediction questionnaire is only the second such questionnaire to be developed specifically for primary care, and is a significant advance on the Berlin questionnaire16 because of its brevity and simple 10-point score. Previously published questionnaires have been derived from sleep clinic populations12–15 and require complex calculations or reference to specialised tables for interpretation. We created the OSA50 questionnaire with the hope that primary care providers would find it ideal for routine clinical use.
The four factors most predictive of OSA were the waist circumference, snoring, witnessed apnoeas and age, with similar variables identified in other studies. Two previously published screening tools include the multivariable apnoea risk (MAP) index13 and the sleep apnoea clinical score (SACS),12 both derived from sleep clinic cohorts. The MAP index is calculated using self-reports of snoring, gasping, witnessed apnoeas, BMI, age and gender.13 The ROC AUC for an AHI ≥10/h was 0.79, with a sensitivity and specificity of 88% and 55%, respectively. Flemons et al12 found the strongest clinical predictors to be neck circumference, hypertension, snoring and partner reports of gasping or choking. For a SACS >15, the authors report a positive likelihood ratio of 5.2 and post-test probability of 81% for an AHI >10/h. Other studies have incorporated craniofacial and oropharyngeal measurements into their prediction models,15 but it is unlikely that primary care providers would adopt a screening tool which required unfamiliar anatomical measurements not routinely used in clinical practice.
Although previously found to be an independent predictor of OSA,12 hypertension was not predictive of OSA in our study. Neck circumference has also been identified as a predictor of OSA.12 Interestingly, we found waist circumference to be a stronger predictor of OSA than other markers of obesity, with at least two other studies reporting similar findings.14 28 This is in keeping with the tracheal traction hypothesis for OSA, in which central obesity is believed to cause a reduction in lung volume during sleep leading to a loss of caudal traction on the upper airway thereby promoting pharyngeal collapse.29 As a questionnaire item, waist circumference is a simpler measure than BMI and is likely to be more familiar to patients than neck circumference. Patients should be aware of their pants size, of which a US men's size of ‘40 inches’ and women's ‘size 16’ are equivalent to the questionnaire cut-off values of >102 cm and >88 cm, respectively. Also, the National Cholesterol Education Program-Adult Treatment Panel (NCEP-ATP III) guidelines have included waist circumference in the diagnostic criteria for the metabolic syndrome using the same cut-off values.30 Primary care physicians are advised to promote intensive lifestyle modifications for patients with metabolic syndrome,31 which is associated with OSA and an independent risk factor for cardiovascular disease.
The diagnostic ability of four clinical prediction formulae, including the MAP index and SACS, was previously evaluated.32 The prediction models had high sensitivities of 76–96% with relatively low specificities of 13–54% and ROC AUC values of 0.67–0.74, similar to the results achieved by our OSA50 questionnaire. While the OSA50 questionnaire alone is not sufficiently accurate to discriminate between those with and without OSA, it is of value in confidently ruling out OSA and directing patients at high risk of disease for further diagnostic evaluation. To improve the accuracy of our diagnostic strategy, we have combined the OSA50 questionnaire with a second simple step of home sleep monitoring. We selected oximetry over nasal pressure for our two-stage model because the failure rate for the oximetry signal (3%) was one-third that of the nasal pressure signal (9%). We also felt that primary care physicians would be more familiar with the concept of oximetry measurement and have greater confidence in their ability to interpret study results. In contrast to laboratory-based PSG, overnight oximetry is less costly and can be self-administered in the patient's home without the need for technical staff. In 2007, the cost to Medicare (Australia's national health insurance system) for PSG was $A485.65 per patient compared with $A106.45 for overnight oximetry. In our validation sample the two-stage diagnostic model accurately identified patients with moderate to severe OSA with a sensitivity and specificity of >80%, with a negative result confidently ruling out disease (NPV 96%). While the PPV appeared lower than ideal (56%), review of the 11 patients classified as ‘false positives’ showed that all had evidence of at least mild OSA (minimum PSG AHI 18.9/h) and half reported excessive daytime sleepiness with ESS scores ≥12 (see appendix 1 in online supplement), and therefore would be likely to obtain benefit from a trial of therapy.
Of note, neither the total ESS score nor individual items related to excessive sleepiness were predictive of OSA, similar to previous reports.12 15 Thus, while daytime sleepiness is a common complaint, it is not per se predictive of disordered breathing during sleep. Hypersomnolence, however, is an important determinant of whether CPAP should be recommended. Patients complaining of excessive sleepiness are more likely to respond to treatment and be compliant with CPAP than those with few symptoms.33 34 Our diagnostic model could be further refined by addition of a minimum ESS score cut-off, thereby specifically targeting hypersomnolent patients who would more likely benefit from urgent treatment.
A two-stage method for OSA using the MAP index and nocturnal oximetry was previously evaluated35 and, for an AHI ≥30/h, had a sensitivity of 85%, specificity of 97%, PPV of 94% and NPV of 92%. Mulgrew et al used a simplified diagnostic algorithm as part of an ambulatory management strategy for moderate to severe OSA which incorporated ESS, SACS and home oximetry.17 Of the 36 patients who scored positively on the diagnostic algorithm, 94% (95% CI 81% to 99%) were correctly identified as having an AHI >15/h on subsequent PSG. These models, however, were evaluated in sleep disorder clinic populations that have a high pretest probability of disease. To our knowledge, our study is the first to evaluate a primary care population where a significant disease burden remains under-recognised.
We attempted to optimise the precision around our estimates by enriching samples with a higher proportion of ‘high-risk’ patients by the Berlin questionnaire, which may raise concerns as to whether our results can be generalised to a ‘non-enriched’ population. Theoretically, changes in disease prevalence do not affect the sensitivity or specificity of a test, but importantly influence predictive values of positive and negative test results. However, by enriching the study population with the Berlin questionnaire, it is possible that high-risk patients with OSA were more likely to score positive responses to selected items on the Berlin questionnaire than an unselected OSA patient sample. This could have influenced the final OSA50 screening questionnaire content and the sensitivity and specificity of the overall diagnostic process. A further effect of enriching a population with cases of moderate to severe OSA may be to falsely elevate the positive predictive value of the two-stage diagnostic screen. As it turned out, however, in our study employment of the ‘enrichment’ strategy had quite different effects in the development and validation samples. It markedly increased the prevalence of moderate to severe OSA in the development group but had little impact on the prevalence of moderate to severe OSA in the validation group since, in the validation group, ‘high-risk’ and ‘low-risk’ patients had similar rates of disease (21% and 18%, respectively). The reasons for this difference are not clear but may relate to the lower age and obesity rate in the validation group compared with the development group. We estimate, using back-extrapolation, that 20% of our original non-enriched primary care population would have had moderate to severe OSA, which is consistent with recent prevalence estimates from the general population.36 The estimated prevalence of moderate to severe OSA in the ‘enriched’ validation group was 21%, which is virtually identical. We therefore believe that estimates of diagnostic accuracy from the validation sample are likely to be similar to that in a non-enriched primary care population. Furthermore, given similar overall rates of moderate to severe OSA in both the high-risk and low-risk groups in the validation sample and face validity of the final questionnaire items, the impact of the potential bias from using an enriched population would appear to be limited.
When applying sensitivity and specificity values obtained from our development and validation groups to a primary care population with a disease prevalence of 20%, the PPV is in the range 55–65% and NPV in the range 97–99%. The two-stage model therefore has its greatest value in ruling out disease. Patients considered ‘false positive’ are likely to have evidence of at least mild OSA (see above) and, if treatment decisions are based on symptoms such as sleepiness, then a trial of CPAP therapy is likely to be worthwhile with minimal risk of harm.
In summary, a two-stage diagnostic model consisting of a screening questionnaire followed by home oximetry can identify patients with clinically significant OSA in a primary care population with a high degree of accuracy. The model could potentially offer a cost-effective solution to the problem of currently overwhelmed laboratory-based sleep services and would be of particular benefit for rural and remote regions, as well as developing countries, where access to sleep services is limited. This simplified strategy, in conjunction with education of primary care physicians and provision of primary care-based management regimens, has the potential to significantly improve patient access to care and expedite treatment for this common sleep disorder.
Acknowledgments
The authors would like to thank the Flinders Medical Centre Clinicians Trust for providing research scholarship support to CLC-C, ResMed for donation of ApneaLink monitors, Amanda McKenna for polysomnography scoring, Amanda Adams for her assistance with database entry and Allison Sulda for distribution of research questionnaires. The authors also wish to acknowledge the support of the following primary care clinics: Belair Medical Clinic, Chandlers Hill Surgery, Flagstaff Hill Medical Centre, Goolwa Medical Centre, Trinity Medical Centre Port Adelaide and Victor Medical Centre.
References
Supplementary materials
Web Only Data thx.2010.152801
Files in this Data Supplement:
Footnotes
Funding This study was funded by the National Health and Medical Research Council of Australia (grant 426744) who have had no role in the design, conduct, collection, management, analysis or interpretation of data, nor have they been involved in the preparation, review or approval of the manuscript for publication.
Competing interests RDMcE has been in receipt of two honoraria for speaking engagements from the sleep device company Philips-Respironics in the last 5 years. RDMcE and NAA have received a grant of $5m from Philips-Respironics for a large randomised controlled trial of CPAP therapy for obstructive sleep apnoea with additional equipment donations from Philips-Respironics, ResMed and Compumedics (all sleep device companies) for this project. Equipment donations have also been received by RDMcE, CLC-C and NAA from Philips-Respironics, ResMed and Somnomed for a randomised controlled trial of simplified management of OSA in specialist and primary care settings. The source of direct funding for the latter two grants was the National Health and Medical Research Council of Australia. The remaining authors have no conflicts of interest.
Ethics approval This study was conducted with the approval of the Repatriation General Hospital and Flinders Medical Centre, South Australia.
Provenance and peer review Not commissioned; externally peer reviewed.