Background Obstructive sleep apnoea (OSA) is highly prevalent in people with spinal cord injury (SCI). Polysomnography (PSG) is the gold-standard diagnostic test for OSA, however PSG is expensive and frequently inaccessible, especially in SCI. A two-stage model, incorporating a questionnaire followed by oximetry, has been found to accurately detect moderate to severe OSA (MS-OSA) in a non-disabled primary care population. This study investigated the accuracy of the two-stage model in chronic tetraplegia using both the original model and a modified version for tetraplegia.
Methods An existing data set of 78 people with tetraplegia was used to modify the original two-stage model. Multivariable analysis identified significant risk factors for inclusion in a new tetraplegia-specific questionnaire. Receiver operating characteristic (ROC) curve analyses of the questionnaires and oximetry established thresholds for diagnosing MS-OSA. The accuracy of both models in diagnosing MS-OSA was prospectively evaluated in 100 participants with chronic tetraplegia across four international SCI units.
Results Injury completeness, sleepiness, self-reported snoring and apnoeas were included in the modified questionnaire, which was highly predictive of MS-OSA (ROC area under the curve 0.87 (95% CI 0.79 to 0.95)). The 3% oxygen desaturation index was also highly predictive (0.93 (0.87–0.98)). The two-stage model with modified questionnaire had a sensitivity and specificity of 83% (66–93) and 88% (75–94) in the development group, and 77% (65–87) and 81% (68–90) in the validation group. Similar results were demonstrated with the original model.
Conclusion Implementation of this simple alternative to full PSG could substantially increase the detection of OSA in patients with tetraplegia and improve access to treatments.
Trial registration number Results, ACTRN12615000896572 (The Australian and New Zealand Clinical Trials Registry) and pre-results, NCT02176928 (clinicaltrials.gov).
- sleep apnoea
- clinical epidemiology
- respiratory measurement
Statistics from Altmetric.com
What is the key question?
Can a two-stage model of questionnaire followed by overnight oximetry accurately detect moderate to severe obstructive sleep apnoea in people with chronic tetraplegia?
What is the bottom line?
This model could substantially increase the detection of obstructive sleep apnoea in people with tetraplegia and subsequently improve access to treatment.
Why read on?
This is the first time a model for detecting obstructive sleep apnoea has been adapted and applied in people with tetraplegia, a population with a high burden of disease but limited access to full diagnostic services.
People with tetraplegia have a higher prevalence of sleep disorders than the non-disabled population.1 The most widely studied sleep disorder in tetraplegia is obstructive sleep apnoea (OSA). More recent estimates of OSA prevalence in chronic tetraplegia range from 56% to 77%2–4 which is higher than in people over the age of 40 without disability (up to 50% in men and 23% in women).5 OSA is associated with both substantial neurocognitive impairment and reduced quality of life in people with tetraplegia.2 6 7 People living with OSA and tetraplegia have a substantially lower health utility value than their tetraplegic peers without OSA. This difference is almost five times the minimally important difference and as such effectively treating OSA is likely to translate into an improved quality of life.2
Current guidelines recommend polysomnography (PSG) for all people with spinal cord injury (SCI) and excessive daytime sleepiness or other symptoms of sleep disordered breathing.8 Full PSG is the ‘gold-standard’ method for diagnosing OSA9 and involves an overnight sleep laboratory stay and connection to a multichannel polygraph during sleep. Very few spinal units have access to PSG, and specific care needs of people with tetraplegia can prohibit access to full PSG in standard sleep laboratories. Even portable PSG requires expensive equipment and specialised staff to apply, score and report the study. Although there are no published estimates on the proportion of people with SCI and OSA who remain undiagnosed, it is likely to be high. It is well recognised that access to PSG is poor and commonly results in long waiting times for diagnosis and subsequent treatment.10 11
The high costs and limited access to PSG have resulted in the development of simpler methods for detecting OSA in people without disability. These simpler methods tend to predict severe OSA with a high degree of accuracy but miss a substantial proportion with milder disease.12–14 More recently, a two-stage model to detect moderate to severe OSA (MS-OSA) has been developed and validated in a primary care, non-disabled population.15 The model, with an overall accuracy of 83%, consists of a simple four-item screening questionnaire (the OSA50) to rule out OSA, followed by overnight oximetry for those with a positive questionnaire result.15
Two OSA screening questionnaires, the Multivariate Apnea Prediction Index and the Berlin Questionnaire, have been tested in the SCI population and both performed poorly at identifying OSA.2 16 OSA is highly prevalent within weeks of cervical SCI and is considered a direct consequence of the injury,17 in contrast to the progressive onset of OSA in people without disability. As such, it is possible that the risk factors also differ, and questionnaires developed specifically for the SCI population may perform better than those developed for the non-disabled population. This project aimed to determine the accuracy of the original two-stage model, developed for the non-disabled population, for diagnosing MS-OSA in people with chronic tetraplegia. Furthermore, the study tested whether inclusion of readily obtainable tetraplegia-specific risk factors would improve model accuracy.
The reference standard for this study, MS-OSA, was defined as an Apnea Hypopnea Index (AHI)≥21, scored with the American Academy of Sleep Medicine (AASM) 2012 criteria. We considered this threshold equivalent to an AHI≥30 scored using AASM ‘Chicago’ criteria; the reference standard used in the original two-stage model validation in a non-disabled population.15 18 19 Further details are provided in online supplementary material including eTable2).
Stage 1: initial validation and modification of the two-stage model with OSA50 questionnaire
An existing data set, including a questionnaire battery and full PSG in a sample of 78 people with chronic tetraplegia, was used to validate and modify the original two-stage model.2 20 Demographic data coupled with questionnaire responses from the Basic Nordic Sleepiness Questionnaire21 enabled derivation of the OSA50 scores (online supplementary eFigure 1 and eTable1). The overnight oximetry was simulated by analysing the raw oximetry signal from the PSG independently of all other signals and scored events. The 3% oxygen desaturation index (3%ODI) was generated by Compumedics (Abbotsford, Vic, Australia) ProfusionPSG software (V.3.4). To mirror the conditions of the original study, the 3%ODI was calculated over total (study) recording time.15 Full PSG data were independently staged and scored as per AASM 2012 criteria to calculate the reference standard AHI.18 Further details are provided in online supplementary material.
Receiver operating characteristic (ROC) curve analyses of the OSA50 and 3%ODI were performed to assess the accuracy of the two stages separately and to determine optimal thresholds. Following application of the model to the data set, sensitivity and specificity, positive and negative predictive values, positive and negative likelihood ratios and overall test accuracy were calculated for the two-stage model as a whole and the 3%ODI alone. This was performed using the original thresholds (ie, OSA50 ≥5/10 and 3%ODI ≥16/hour) and repeated with the optimised thresholds.
A 95% CI for the 3%ODI threshold was calculated by obtaining 999 bootstrap replicate samples.22 Resampling separately the ‘OSA negative’ and ‘OSA positive’ cases and determining the optimal threshold for each sample provided the 95% CI as the 2.5th and 97.5th percentiles of the 999 thresholds.
Stage 2: development of the tetraplegia-specific questionnaire (Screening for OSA in Tetraplegia) and two-stage model
Using the same data set (n=78) a modified version of the questionnaire was developed by investigating previously identified, tetraplegia-specific risk factors and their associations with MS-OSA. The risk factors investigated were age, gender, American Spinal Injury Association (ASIA) Impairment Scale (AIS), lesion level, neck and waist circumference, body mass index, time since injury, daytime sleepiness, self-reported snoring and self-reported apnoeas. Non-binary variables were dichotomised to enable simple questionnaire administration. Further details are provided in online supplementary material.
Univariate associations between the binary risk factors and an AHI≥21 were investigated. Variables with a p<0.1 on univariate analysis were entered into a backward, stepwise, multivariable logistic regression model. Weightings of the regression coefficients of variables significantly associated with MS-OSA (p<0.05) were used to develop a simple scoring algorithm for a new questionnaire, called Screening for OSA in Tetraplegia (SOSAT).23
The same diagnostic accuracy statistics used in stage 1 were calculated for the SOSAT questionnaire alone and, after inclusion of the 3%ODI, for the two-stage model as a whole.
Stage 3: validation of two-stage models
Stage 3 involved validation of the two-stage models (with both OSA50 and SOSAT screening questionnaires) against the reference standard (PSG derived AHI≥21) in a prospective sample. Study design complied with the Standards for Reporting of Diagnostic Accuracy Studies (STARD) statement .24 The study was prospectively registered in the Australia and New Zealand Clinical Trials Registry (ACTRN12615000896572). The University of Miami provided baseline data from a concurrent randomised controlled trial which was prospectively registered on clinicaltrials.gov (NCT02176928).
The sample size calculation for the prospective validation was based on the estimated sensitivity, given its relative importance for diagnosis of this disease. Assuming a 50% prevalence of MS-OSA,2 4 98 participants were required for a sensitivity of 0.85 (95% CI 0.75 to 0.95). Assuming a home sleep study failure rate of approximately 9%,15 we aimed to recruit 108 participants.
Consecutive patients, with chronic (>1 year postinjury), traumatic tetraplegia (level T1 or higher; AIS A, B, C or D), attending the spinal outpatient or inpatient units between September 2015 and April 2017 at Austin Hospital, Stoke-Mandeville Hospital and GF Strong Rehabilitation Centre were invited to participate. All participants with chronic, traumatic tetraplegia recruited to the University of Miami study between April 2015 and November 2016 were also assessed for inclusion. Participants were excluded if they were: being treated for OSA; an inpatient with a cardiorespiratory complication; medically unstable; or unable to provide informed consent.
Unattended PSGs were conducted in the participants’ homes or the spinal inpatient units and set up by two trained staff. PSGs were conducted with a SomtePSG device (Compumedics, Abbottsford, Australia) except in Miami where an Embla Emblettax100 PSG device (Natus Medical, Pleasanton, USA) was used. All studies were sleep staged, and respiratory scored by an independent, experienced sleep scientist in Melbourne using Profusion software (Compumedics).
Oximetry was collected within the PSG devices and the 3%ODI generated as described in stage 1. To determine whether a different oximeter could be confidently used in the model, oximetry was also collected with a ResMed (San Diego, CA, USA) ApneaLinkAir device in a subset of 20 participants, and the 3%ODI was calculated using ApneaLink software (V.10.20). The ApneaLink oximeter was placed on the same finger of the opposite hand to the SomtePSG oximeter. Detailed descriptions of PSG devices and oximeters are provided in online supplementary material.
Demographic data were collected from the medical record. Abdominal girth (at end expiration) and neck circumference were measured immediately prior to the sleep study with the patient in the supine position. If weight and height were not recorded in the medical record, participants provided estimates. The Berlin Questionnaire25 and Karolinska Sleepiness Scale26 were collected before the sleep study.
Baseline characteristics of participants in the development and the validation groups were compared with Student’s t-tests (or Wilcoxon signed-rank test if non-normally distributed) and Χ2 analyses.
The accuracy of both two-stage models and the ODI alone were evaluated with the same diagnostic accuracy statistics described previously. Differences in overall accuracy of the models were evaluated with a McNemar Χ2 test.
Sensitivity analyses of model accuracy examined the potential effects of study site (Χ2) and PSG quality. High-quality sleep studies were defined as 3 hours of sleep plus 6 hours of concurrent EEG, oxygen saturation and either nasal flow and/or thoracic/abdominal excursion traces.
Agreement between the 3%ODIs generated by both the ApneaLink and Compumedics devices was compared using a Bland-Altman plot, Pearson’s correlation and the proportion changing categories.
Participants of the development (n=78) and validation (n=100) groups were predominantly male, slightly overweight and less than half had complete injuries (AIS A). Participants of the validation group were on average 6 years older and 6 years longer postinjury than those in the development group. Prevalence of OSA was high in both groups, although significantly higher in the validation group (table 1). See online supplementary material for the characteristics of sleep disordered breathing in both samples.
Stage 1: initial validation and modification of two-stage model with OSA50 questionnaire
In the development data set, the OSA50 questionnaire alone was significantly predictive of MS-OSA with an ROC area under the curve (AUC) of 0.83 (95% CI 0.73 to 0.92, figure 1). When using the original threshold (≥5/10), the sensitivity and specificity of the questionnaire were 86.7% and 52.1%, respectively. A threshold of ≥3/10 with a sensitivity of 100% and a specificity of 29.2% gave the best performance for ruling out OSA.
The ROC AUC for the 3%ODI alone was 0.93 (0.87–0.98, figure 1). When using the original threshold of ≥16, sensitivity and specificity of the ODI were 80.0% and 87.5%. A threshold of ≥13 was deemed the optimal threshold for classifying MS-OSA (86.7% and 83.3%, respectively), and the 95% CI was 9.5 to 22.2.
Sensitivity and specificity of the two-stage model (OSA50 questionnaire and ODI in combination) with original thresholds (OSA50 ≥5/10 and 3%ODI ≥16) were 70.0% and 91.7%. Sensitivity and specificity with optimised thresholds (OSA50 ≥3/10 and 3%ODI ≥13) were 83.3% and 85.4% (table 2 and online supplementary eTable 3A–C).
Stage 2: development of the tetraplegia-specific questionnaire (SOSAT) and two-stage model
The ROC analyses for non-binary predictor variables, the thresholds selected and the subsequent univariate analyses can be found in online supplementary eTables 4 and 5. Multivariable analysis established that four variables were predictive of MS-OSA (table 3). The multivariable factor weightings were simplified to generate a scoring algorithm out of 10 for the SOSAT questionnaire (figure 2). Two variables were given a weighting of 3 (AIS A, B or C and self-reported snoring) and two variables a weighting of 2 (self-reported apnoeas and sleepiness).
The ROC AUC for the SOSAT questionnaire was 0.87 (0.79–0.95, figure 1). Sensitivity and specificity at a threshold of ≥5/10 were 100.0% and 27.1%, respectively. When combined with oximetry the sensitivity and specificity of the two-stage model (SOSAT ≥5/10 and 3%ODI ≥13) were 83.3% and 87.5% (table 2 and online supplementary eTable 6).
Stage 3: validation of two-stage models
Figure 3 shows the participant recruitment pathway for the validation group. Three models were applied to the validation data set for comparison: the two-stage model with OSA50 (with optimised thresholds), the two-stage model with SOSAT and the 3%ODI alone (tables 2 and 4A–C). No differences in the overall accuracy of the three models were observed (p=0.56, 0.48, 0.56 for the three pairwise comparisons). The OSA50 and SOSAT questionnaires excluded 19% and 22%, respectively, from further testing with oximetry.
The SOSAT questionnaire incorrectly excluded three people with MS-OSA and 3%ODI ≥13. In the model using SOSAT, the 3%ODI was responsible for 18 of the 21 incorrectly classified cases. Of these, eight (44%) were within the 95% CI for the 3%ODI (9.5 to 22.2). The AHIs for the 12 incorrectly missed and the nine incorrectly diagnosed ranged from 21.5 to 37.9, and 12.6 to 20.3 respectively (online supplementary eTable 7).
There was no effect of study site (model with SOSAT p=0.40; OSA50 p=0.54), nor PSG quality on overall accuracy (high quality (n=76) vs all (n=100); model with SOSAT=79% vs 79%; with OSA50=82% vs 80%).
Oximetry device data comparisons
Oximetry data from simultaneously collected ApneaLink and Compumedics devices revealed four technical failures when using the ApneaLink. Mean 3%ODI scores from the 16 participants with both ApneaLink and Compumedics devices were 20.0 (SD=16.7) and 21.6 (17.2) (p=0.36). Correlation was high (0.92; p<0.01). Two (12.5%) participants whose Compumedics derived 3%ODI was ≥13 were <13 on ApneaLink (online supplementary eFigure2 for Bland-Altman plot).
This study evaluated the accuracy of a two-stage model of a screening questionnaire followed by overnight oximetry as an alternative to PSG for diagnosing MS-OSA in people with chronic tetraplegia. The model was tested using both the OSA50 questionnaire, as originally developed and validated for the non-disabled primary care population15 and a modified version developed specifically for chronic tetraplegia. To our knowledge, this is the first time an OSA diagnostic model has been adapted and applied in people with tetraplegia, a population with a high burden of disease but limited access to full diagnostic services. Both models performed similarly, correctly classifying 80% (OSA50) and 79% (SOSAT) of participants.
We hypothesised that the model with SOSAT questionnaire would be more accurate than with OSA50 questionnaire. However, two of the four items (self-reported snoring and apnoeas) were identical. Despite the strong association found on univariate analysis, waist circumference (an item of the OSA50) was excluded from the SOSAT questionnaire because it is a highly impractical measurement to obtain in tetraplegia. The absence of waist circumference may partly explain why SOSAT did not outperform the OSA50. Given the performance of SOSAT was comparable to the OSA50 with simpler administration we recommend using SOSAT.
The two-stage model with SOSAT incorrectly diagnosed nine participants with MS-OSA. Their AHIs ranged from 12.6 to 20.3. All had evidence of at least mild OSA and would potentially benefit from therapy. Of greater concern were the 12 participants who were incorrectly ‘missed’ in the model (AHIs 21.5–37.9). In six of these cases the SOSAT questionnaire failed to identify individuals at risk, and in nine the ODI was too low. In settings where PSG is readily available, clinicians could order full PSG for patients whose unexplained symptoms persist, despite the negative result in this model.
Despite sensitivity and specificity results that were comparable to the original Chai-Coetzer et al’s validation study,15 the negative predictive value (76%) is low in our sample, indicating a 24% chance of a false-negative result. The high prevalence of MS-OSA in this population poses challenges for any diagnostic model to modify post-test probability. The use of CIs partially addresses this issue. A bootstrapping technique determined the optimal threshold for the 3%ODI was ≥13 with a 95% CI of 9.5 to 22.2. Application of the CI could reduce the risk of misclassification. We suggest that clinicians use the 95% CI for the 3%ODI as an ‘uncertain’ category where, with clinical judgement, they could further investigate OSA and/or proceed to treatment, based on the symptoms and desires of the individual patient. In our study, removing the incorrectly classified participants with 3%ODIs within the CI would have improved the sensitivity and specificity of the two-stage model with SOSAT to 80% and 93%, with an overall accuracy of 86% (n=92).
When OSA is defined as AHI≥5, the prevalence in our validation sample was found to be 97%. This is substantially higher than in the retrospective sample used to develop the model (73%), yet similar to that recently reported in a prevalence study using comparable methods (93%).4 Unfortunately, methodologies vary substantially among prior prevalence studies and as such, there are no meta-analyses that estimate the population prevalence of OSA in SCI.1 Undertaking screening tests for a highly prevalent disease could be considered redundant. However, funding for treatment is usually dependent on a clinician’s diagnosis of OSA, and few clinicians and patients would prescribe and accept treatment without solid evidence of disease. We have focused on detecting MS-OSA, for which the prevalence was 38% and 53% in our two samples. In people without disability, all-cause mortality, stroke and cardiovascular disease are strongly associated with more severe OSA.27–30 In people with acute and chronic tetraplegia, more severe OSA has been associated with worse neuropsychological function.6 7 Further, those with more severe disease and more daytime sleepiness are more likely to adhere to CPAP treatment, the first-line therapy for OSA.31
There is currently no agreed threshold for the diagnosis of OSA or its severity9 and furthermore, the AHI is poorly correlated with symptoms.32 There are also significant problems with intra-rater and inter-rater reliability of AHI scoring and different laboratories use different scoring rules for hypopnoeas which substantially impact the AHI.9 Reliability studies have demonstrated high night-to-night variability in the AHI, affecting OSA diagnosis at various thresholds.33 34 Other research suggests the ODI has relatively low variability35 and a comparison of the night-to-night variability of respiratory sleep indices found that the ODI was substantially more reliable than the AHI.36 Additionally, the ODI, and not the AHI, has been significantly associated with cardiovascular disease in a large non-disabled population with suspected OSA.37 This evidence suggests that ODI is a more reliable and, potentially, a better marker of cardiovascular risk than the ‘gold-standard’ AHI. Researchers and clinicians must be mindful of the limitations of the AHI for diagnosing OSA, and the implications for diagnostic accuracy studies investigating alternative models. Evaluating the effectiveness of alternative tests on patient outcomes rather than traditional diagnostic accuracy methods has been suggested to address this issue.9
While central sleep apnoea has previously been identified in tetraplegia,38 our data suggest that it is not the predominant characteristic of sleep disordered breathing, accounting for just 4% of the classified events on average. Only 2% of our combined sample (3/178) had predominant central sleep apnoea. Hypoventilation is a risk in patients with neuromuscular weakness, however its frequency and severity in tetraplegia is yet to be established. Assessment for hypoventilation would be an important component of any OSA management pathway using our two-stage model. Further research is required to assess the safety and feasibility of using this model in an OSA management pathway. Consideration of symptoms, individual patient comorbidities, assessment for hypoventilation and the availability of full PSG for ambiguous or complicated cases would also be important components of this pathway.
The scope of our study was limited to testing the accuracy of a two-stage model of questionnaire followed by overnight oximetry (a level IV portable device). As yet, the accuracy of level III portable monitors, including airflow, respiratory effort and oximetry recordings, has not been tested in people with tetraplegia, although they have been assessed as feasible in this population.39 Level III monitors are an accepted alternative for OSA diagnosis in non-disabled populations.40 Based on research showing similar effectiveness of level III devices to PSG and substantially lower costs, others have suggested that PSG is no longer necessary for most patients with clinical suspicion of OSA.41 Further research into the accuracy of level III devices and their safety and acceptability in tetraplegia is warranted.
The ODIs were obtained from the PSG oximeter and the Compumedics software. A recent study comparing the ODIs generated by a ResMed system with those from a Compumedics system found the ResMed system generated higher values. The source of the difference was in the data collection processes rather than the algorithms built into the software.42 This finding has major implications for the use of ODIs to diagnose OSA and may limit translation of models using oximetry into clinical practice. It highlights a sleep industry-wide failure to develop reference criteria for oximeters, analysis software and algorithms. Our addition of the ODI CI to the two-stage model partially addresses this issue. In our substudy there were two participants with conflicting OSA classifications from the ApneaLink and Compumedics systems. In both cases, the wrongly classified ODI was within the CI.
The development and validation samples in this study were recruited with different methods and approximately 10 years apart. The validation group were older, longer postinjury and had more severe OSA, which may reflect a selection bias and have affected the performance of the model.
The two-stage model of SOSAT questionnaire followed by overnight oximetry provides a potential alternative to full PSG for identifying MS-OSA in people with chronic tetraplegia. This model could be considered in settings where PSG is inaccessible or when patients are unable or unwilling to attend an overnight sleep study, in conjunction with comprehensive assessment of symptoms, comorbidities and hypoventilation. Some patients with MS-OSA may be missed; however, the addition of CIs to the 3%ODI may reduce this risk. Despite the limitations, this translatable model has the potential to substantially increase the detection of OSA in people with tetraplegia and subsequently improve access to treatment.
The authors thank ResMed for the donation of the ApneaLink monitors, and the research staff involved in the recruitment of participants and collection of data. We also thank our participants with spinal cord injury who so generously allowed us into their homes to perform the sleep studies.
Contributors Study design: MG, RS, SS, NTA, MSN, WRR, CLCC, PR, TC, SG, DJB. Participant recruitment and data acquisition: MG, RS, SC, CT, SS, VV, WRR. Data analysis: MG, RS, WRR, TC, PR, DJB. Manuscript preparation and approval of final version: all authors. Guarantors: MG and DJB.
Funding MG was supported by an Australian Government National Health and Medical Research Council postgraduate scholarship (grant number 1114181) and an Australasian Spinal Cord Injury Network PhD scholarship. Data collection at Stoke-Mandeville Hospital was supported by a Stoke-Mandeville Masson Research Award grant from the Buckinghamshire NHS Trust Charitable Trust Funds, UK. Data collection at the University of Miami was funded by the Department of Defense (Award No W81XWH-13-1-0479).
Disclaimer None of the funders have had any role in the design, conduct, collection, management, analysis or interpretation of the data, nor have they been involved in the preparation, review or approval of the manuscript for publication.
Competing interests None declared.
Patient consent Obtained
Ethics approval Austin Health, Melbourne; GF Strong, Vancouver; NHS Health Research Authority, UK; The University of Miami, Miami.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.