Background One in three patients hospitalised due to acute exacerbation of COPD (AECOPD) is readmitted within 90 days. No tool has been developed specifically in this population to predict readmission or death. Clinicians are unable to identify patients at particular risk, yet resources to prevent readmission are allocated based on clinical judgement.
Methods In participating hospitals, consecutive admissions of patients with AECOPD were identified by screening wards and reviewing coding records. A tool to predict 90-day readmission or death without readmission was developed in two hospitals (the derivation cohort) and validated in: (a) the same hospitals at a later timeframe (internal validation cohort) and (b) four further UK hospitals (external validation cohort). Performance was compared with ADO, BODEX, CODEX, DOSE and LACE scores.
Results Of 2417 patients, 936 were readmitted or died within 90 days of discharge. The five independent variables in the final model were: Previous admissions, eMRCD score, Age, Right-sided heart failure and Left-sided heart failure (PEARL). The PEARL score was consistently discriminative and accurate with a c-statistic of 0.73, 0.68 and 0.70 in the derivation, internal validation and external validation cohorts. Higher PEARL scores were associated with a shorter time to readmission.
Conclusions The PEARL score is a simple tool that can effectively stratify patients' risk of 90-day readmission or death, which could help guide readmission avoidance strategies within the clinical and research setting. It is superior to other scores that have been used in this population.
Trial registration number UKCRN ID 14214.
- COPD Exacerbations
- COPD epidemiology
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/
Statistics from Altmetric.com
What is the key question?
In patients admitted to hospital with an exacerbation of COPD, what predicts 90-day readmission or death?
What is the bottom line?
The PEARL prognostic score comprises five indices: Previous admissions, eMRCD score, Age, Right-sided heart failure and Left-sided heart failure, and is a consistent and accurate predictor of 90-day readmission or death.
Why read on?
Currently, postdischarge care is directed by clinical judgement, although clinical judgement alone is suboptimal: the PEARL score is easy to apply at the bedside using indices that are routinely available in all patients, accurately risk stratifies patients according to risk of readmission and may inform postdischarge planning in patients admitted with acute exacerbation of COPD.
Acute exacerbation of COPD (AECOPD) is one of the most common reasons for hospital admission and one-third of patients are readmitted within 90 days.1 ,2 Clinicians are poor at identifying patients at particular risk of readmission.3 A simple and accurate prognostic tool would help identify those who may benefit most from additional health and social care services, and this hopefully will translate into improved outcomes for patients and more efficient use of scarce resources. A condition-specific score to predict this outcome following hospitalisation for AECOPD has not been developed.
Within the CODEX study,4 the ADO, BODEX, CODEX and DOSE scores were all assessed for their performance at predicting the combined outcome of readmission or death without readmission in patients with COPD (see table 1). However, most were originally developed to predict death (ADO5 and BODEX6) or health status (DOSE7) and all offer only modest prediction of readmission/death without readmission. The performance of CODEX was superior to the updated ADO and BODEX score, with a trend towards better performance than DOSE, but this comparison was in the CODEX derivation population.4 Derivation studies bias results in favour of the derived tool, in this instance CODEX, and so comparison in an external validation cohort is required.
LACE8 was primarily derived to assess readmission/death without readmission in unselected emergency and elective admissions, and has not been compared with the other scores in table 1, which are more COPD specific. It offers modest performance, and exceeds alternative generic scores that have undergone validation.3 ,9 A generic score such as LACE is preferable to a disease-specific score, unless the latter offers superior performance.
In common with previous studies, we selected ‘death without readmission’ as a combined primary outcome with ‘readmission’. This is justified as patients who die without readmission are likely to have been readmitted had the clinical deterioration been recognised in time, and death without readmission and readmission share similar predictors.
We selected a 90-day timeframe for our primary outcome as this covers the high-risk period. In patients hospitalised due to AECOPD who survive to discharge, one-third are readmitted within 90 days2 and the risk of further exacerbation and readmission over the 8–12 weeks postdischarge outweighs the risk over the subsequent year.10 ,11 Such events are associated with substantial risk of death, adverse qualify of life12–15 and high healthcare costs.16
The aims of the present study were:
To develop and perform internal and external validation of a tool to predict 90-day readmission or death without readmission in patients discharged from hospital following admission with AECOPD.
To assess tool performance at 30 days, and compare the new tool with other prognostic scores (table 1) at 30 and 90 days.
Study design and participation
The study populations comprise consecutive patients admitted with AECOPD within the DECAF derivation17 and validation18 cohorts who survived to discharge. Initially, a prediction model was developed in two UK hospitals (the derivation cohort); the performance of this model was then assessed in: (a) the same hospitals, but over a later time frame (‘internal validation cohort’; temporal validation) and (b) four further UK hospitals (‘external validation cohort’; temporal and geographical validation). External sites were chosen to ensure variation in COPD prevalence, socioeconomic factors, rurality and structure of care. This readmission study was a prespecified aim of this programme of research.17 ,18
Patients in the derivation and external validation cohorts were recruited prospectively, with review of coding records to maximise capture. In the internal validation cohort, patients were identified retrospectively from a broad coding records search and review of COPD assessments routinely completed by the specialist nursing team. Patients in the derivation study were not excluded. When collecting candidate indices researchers were unaware of outcome. The internal and external validation cohorts were individually powered.
Inclusion criteria were: a primary diagnosis of an exacerbation of COPD; spirometric evidence of airflow obstruction; age 35 years or older and smoking history of 10 or more cigarette pack-years. Exclusion criteria were: previous inclusion in the same cohort, and any illness (other than COPD) likely to limit survival to <1 year.
Most sociodemographic and clinical data were collected at the time of admission; additional indices collected up to the point of discharge included length of stay and treatment with non-invasive ventilation. Candidate predictors included stable demographic, clinical and functional indices, including markers of frailty. The 90-day readmission and mortality data were collected from medical records. The selection of potential predictors of readmission/death without readmission was informed by a systematic literature search and clinical plausibility.19 Those predictors that are not routinely assessed or available at admission were appropriately excluded to avoid bias from missing data and to ensure both generalisability and ease of application of the final tool.20 ,21 When two or more indices were closely related, the most appropriate was selected (collinearity—see the Statistical methods section). Final candidate indices assessed are shown in table 2.
In prognostic research, it is important that both the definitions of indices and methods of measurement are clear.21 All sites received data collection guides, which included information on sources of data, the definitions of diseases and terms and time-points for collection. We assessed stable state breathlessness using the extended MRC dyspnoea score (eMRCD). This includes a measure of frailty (ability to wash and dress independently).18 eMRCD assesses breathlessness ‘on a good day’ within the previous 3 months. Key differences to the traditional tool include: (a) use of the term ‘unable to leave the house unassisted’ as opposed to ‘house bound’; (b) division of the most severe category into eMRCD 5a and 5b patients, depending on whether the person requires help with both washing and dressing and (c) clear transition between levels.
Left ventricular failure (LVF) referred to a preadmission diagnosis based on echocardiograph as per the European Society of Cardiology guidelines.22 Three patients had LVF confirmed by a different modality (CT cardiac angiography n=1, cardiac MRI n=2).
For cor pulmonale, a clinical diagnosis was accepted, in the absence of an echocardiograph. First, this allowed for a new diagnosis of cor pulmonale at admission based on clinical assessment alone, which is consistent with usual clinical practice. Second, echocardiograph alone lacks precision to diagnose cor pulmonale in comparison with heart catheterisation, which is the gold standard.23 There were 193 patients who had a diagnosis of cor pulmonale, of whom 62 had preadmission echocardiographs showing cor pulmonale. Cor pulmonale on echocardiograph was defined as right ventricular impairment or a raised pulmonary artery systolic pressure of 35 mm Hg or more in association with lung disease.
Admission and readmission were defined as an admission to a hospital ward outwith the emergency department. Clinical care was not influenced by the research team, and no additional tests were performed.
The derivation cohort was adequately powered based on the minimum expected events per index (events per index: recommended=10 or more; observed=14).24 The available populations from the DECAF internal and external validation cohorts were individually powered on mortality. Given the higher event rate for readmission/death without readmission, both the internal and external validation cohorts were robustly powered, with a sample size of 227 required for an expected sensitivity of 70%, an SE for this estimate of 5% and an event rate of 37%.25 Baseline population characteristics and outcome were described using proportions, means with SDs or medians with IQRs, and compared using Fisher's exact test, analysis of variance and Kruskal-Wallis test. Logistic regression models were compared with Akaike Information Criterion (AIC).
Multiple imputation was used for predictor indices with <20% missing data, using IBM SPSS statistics V.22 to create five datasets and results were pooled using Rubin's method.26 Data were imputed using the Markov Chain Monte Carlo method, with linear and logistic regression for continuous and categorical indices. There was no missing data for outcome. Regression analysis between predictors and missing data supported the assumption that data were missing at random.20 ,27 A large number of indices (n=67) were included in the imputation model. Multicollinearity between predictor indices was addressed28 and indices were dichotomised or categorised by visual inspection of the receiver operating characteristic (ROC) curve, a clinically relevant cut-off, or a median split (see online supplementary figure E1).17 To develop the final model, logistic regression using backward elimination was performed, with a probability of 0.1 for exclusion,29 and scores were assigned based on their regression coefficients.30 As prediction is about estimation, it is reasonable to include predictors with p values >0.05, otherwise strong predictors that are rare may be inappropriately excluded.20 The discrimination of the model was assessed by measurement of the area under the ROC (AUROC) curve in each cohort, and compared with other clinical scores (table 1) by the method of DeLong et al with and without multiple imputation.31
Calibration was assessed by the Hosmer-Lemeshow goodness-of-fit test,32 by comparing the full regression model to the weighted model, and by comparing outcomes and re-examining the score assigned to individual indices, across all three cohorts. Kaplan-Meier and the log rank test were used for time to events.
Analyses were performed using IBM SPSS statistics V.22 and SigmaPlot V.12.3.
Of the candidate indices shown in table 2, all had 1% or less missing data for each cohort except for pH (derivation 6.6%, internal validation 9.4% and external validation 16.2%), weight loss (derivation 2.9%, internal validation 1.4%, external validation 12.4%), admissions per year (derivation 0%, internal validation 0%, external validation 2.1%) and cough effectiveness (derivation 0%, internal validation 0.37%, external validation 1.5%). Missing data for all indices, prognostic scores and patients are shown in online supplementary table E1.
In the derivation, internal validation and external validation cohorts, 824 (December 2008–June 2010), 802 (January 2012–May 2013) and 791 (April 2013–May 2014) patients survived to hospital discharge, of whom 309 (37.5%), 297 (37.0%) and 330 (41.7%) were readmitted or died within 90 days of discharge. The population characteristics of each cohort are shown and compared in table 2. A diverse population of patients with AECOPD were recruited, exemplified by significant differences between cohorts in median eMRCD score, admissions in the previous year and proportions with LVF and cor pulmonale. The population characteristics of individual hospitals are described elsewhere.17 ,18
Development of a predictive score
The following indices were categorised: age <80 or 80+; cigarette pack-years <45 or 45+; eMRCD score 1–3, 4, 5a or 5b; FEV1 %predicted <50 or 50+; previous admissions (<2 or 2+ in the past year) and length of stay as per the LACE study (0, 1, 2, 3, 4–6, 7–13, or 14+ days).8 All candidate indices were analysed using backwards multivariate logistic regression (table 2). Weight loss was not entered into the model due to the high rate of missing data. Indices with high missing data rates may provide biased estimates as the test may only be performed in select patients; furthermore, collecting this index was labour intensive, a problem which would likely recur in clinical practice.29
The indices retained in the final model were: Previous admissions, eMRCD score, Age 80 or more, cor pulmonale (‘Right ventricular failure’) and Left ventricular failure, and were collectively named the PEARL score (table 3, and see online supplementary table E2). The PEARL regression equation was compared within the derivation cohort by AIC, entering age as a categorical and continuous variable: PEARLage catergorical AIC=940.4, PEARLage continuous AIC=940.8. This is the recommended approach to compare the relative quality of two related models,21 and shows no difference, supporting the categorisation of age for ease of application of the score.
As continuous variables were dichotomised, primarily to ensure ease of use, the regression coefficients (the column entitled ‘B’ in table 3) show the relative contribution of each index. The coefficients were used to assign initial weights to each index in the derivation cohort. There are various approaches to adjust models to improve prediction and generalisability27 ,33; this can involve combining derivation and validation data sets.27 The assigned weights were re-evaluated after pooling all three cohorts. The original weightings assigned in the derivation cohort were appropriate, except ‘previous admissions’, which should optimally be weighted as three (table 3).
Performance and calibration of the PEARL score
The AUROC for the PEARL score for 90-day readmission/death without readmission was: derivation=0.73 (95% CI 0.70 to 0.77); internal validation=0.68 (95% CI 0.64 to 0.72) and external validation=0.70 (95% CI 0.66 to 0.73). In all three cohorts combined, the AUROC for 90-day readmission/death without readmission was 0.70 (95% CI 0.68 to 0.73). For 90-day readmission, only (not including death) the AUROC was 0.69 (95% CI 0.67 to 0.71).
The risk of readmission or postdischarge death increases with higher PEARL scores (table 4). Further details of all cohorts, and data on readmission and death as lone outcome are shown in the online supplementary table E3; across all three cohorts, risk was similar with all p values >0.05 showing that predictions are consistent. We grouped scores into low-risk (0–1), intermediate-risk (2–4) and high-risk (5+) PEARL scores. Sensitivity and 1–specificity for the PEARL score are shown in the online supplementary table E4. In the low-risk group (PEARL 0–1), only 2.5% (22/890) died postdischarge within 90 days.
Calibration was further assessed by plotting ‘expected probability’ (calculated from the full regression equation) against the ‘observed probability’. Calibration tends to perform best in derivation cohorts, so the derivation and validation cohorts were plotted separately. Again, PEARL was well calibrated (perfect calibration would fall on the 45° line; see figure 1).
Comparison with other prognostic scores
The ROC curves for each prognostic score are shown in figure 2 for the validation cohorts combined. Comparison within a derivation cohort favours the developed tool, so the derivation cohort is not included within the graph. This shows that the ROC curve area is higher for PEARL than the other scores.
Table 5 shows the comparison between PEARL and all other tools for each individual cohort. PEARL was superior to ADO, BODEX, DOSE and LACE in all three cohorts, and to CODEX within the derivation and external validation cohorts. Results were unchanged with complete case analysis. Thirty-day comparisons are shown online (see online supplementary supplementary table E6).
Time to death or readmission, and readmission frequency
Time to death or readmission was available for 90 days in all three cohorts and for 1 year in the derivation and internal validation cohorts. Higher PEARL risk groups were associated with a shorter time to death or readmission (figure 3 and see online supplementary table E7).
PEARL risk group identifies those at risk of frequent admissions. For risk groups 0–1, 2–4 and 5–9, the median (IQR) number of readmission was 0 (0–1), 1 (0–2) and 2 (1–3). When adjusted for death (time exposed to readmission), the risk was 0 (0–1.8), 1 (0–3) and 3 (1–6), respectively.
We have developed and validated a model to predict 90-day readmission/death without readmission in patients hospitalised with an AECOPD, the ‘PEARL score’. The tool was designed to be easily applied at the bedside using indices routinely available at admission, and performance was superior to alternative scores. The risk of readmission/readmission without death was considerably higher in the first 90 days than during the rest of the year, both overall and within the moderate-risk and high-risk PEARL groups, which justifies our chosen timeframe. Rates of readmission were similar to those seen in the European National Audit 2016.34 Our composite end point is more appropriate than readmission alone, as the latter would include both those who are neither readmitted nor die and those who die without readmission in the ‘favourable’ outcome group. Accurate risk stratification of patients should help efficiently direct resources aimed to reduce readmissions, such as supported discharge services, pulmonary rehabilitation, education programmes and possibly azithromycin therapy, although the impact of these strategies requires assessment. Furthermore, identification of patients who are at risk of death without readmission may allow services to be put in place to facilitate early recognition of deterioration and readmission.
The study has a number of strengths, most importantly consecutive recruitment of patients and high case ascertainment. This is supported by excellent recruitment rates at all sites which were substantially higher than the 2015 UK national COPD audit, as detailed previously.18 Generalisability is supported by consistent performance in three cohorts, including the prospective external validation cohort (the ‘gold standard’ for assessing performance). The six hospitals that took part had different structures of care and varied populations, with respect to readmission avoidance schemes, COPD prevalence, socioeconomics and rurality. Furthermore, data were collected by a variety of healthcare professionals, including physicians and specialist nurses.
The eligibility criteria were inclusive; few patients were excluded due to poor prognosis (expected survival <1 year for an illness other than COPD): in the internal validation cohort, for example, this comprised only 27 patients (3.4%), principally due to metastatic malignancy. Definitions were aligned with usual clinical practice, and were pragmatic to reduce missing data and the consequent risk of bias. This approach is regarded as a key strength in prognostic research.20 ,21 Primary outcome data were available in all patients, missing data rates were low and multiple imputation and complete case analyses showed that results were robust. Further study strengths can be seen in the online supplement, ‘The CHARMS checklist’,21 which provides a framework to critique prognostic studies.
There are a number of limitations within the study. Most patients in the internal validation cohort were identified retrospectively, which may have compromised performance. However, the risk of any consequent bias is probably low as the relevant indices were recorded during the patients' admission prior to the outcome. When extracting data, researchers were blind to outcome. Furthermore, the external validation cohort was prospective and individually powered.
Varied rates of cor pulmonale and LVF may reflect inconsistency in clinical assessment, or could represent true population differences, supporting external validity. Despite variation in rates, both were associated with readmission/death without readmission in all three cohorts. Dichotomising continuous indices, such as age, allows a score to be calculated at the bedside without the full regression equation and a computer. When this approach is adopted, concern about consequent loss of prognostic strength may be raised. However, the impact of dichotomising (or categorising) continuous variables on performance may be minimal if the relation between the index and risk of outcome is non-linear, and the prognostic threshold(s) appropriately selected.20 The impact of dichotomising age on the performance of the PEARL score was negligible. We did not differentiate between different causes of readmission such as respiratory or non-respiratory. This would require the derivation and validation of separate scores, and based on data from our validation cohorts most admissions are respiratory (more than three in four). Furthermore, differentiating between cardiac and respiratory causes of readmission can be challenging. While PEARL performed well in all cohorts, and participating units were chosen to ensure variation in population and structure of care, confirmation of performance in healthcare settings outside the UK is desirable. A final limitation, which is common to all prognostic research, is that the strength of the association of the predictor with the outcome may vary between patients.35
Other prognostic research shows that accurate prediction of readmission is challenging.9 The discrimination of PEARL (external validation AUROC=0.70) is superior to all other tools that predict readmission or death in AECOPD, and substantially stronger than clinical judgement (AUROC=0.56–0.59).3 An extensive literature search was performed to ensure the inclusion of all potential predictor variables that could be easily collected at the bedside. All of the indices in the PEARL score have been previously shown to predict our outcome, except for eMRCD score. This is important as eMRCD (along with previous admissions) was the strongest predictor. In our study, associated pneumonia, non-invasive ventilation and institutional care (nursing or residential home) did not appear in the final model. This does not mean that they are not predictors, but rather that they did not add prognostic power to the PEARL indices, which were stronger predictors. Furthermore, eMRCD includes a measure of frailty which may, at least in part, capture the risk associated with such indices. The strongest predictors of readmission/death without readmission tended to be measures of underlying disease severity, frailty and comorbidity rather than measures related to the acute event. For instance, DECAF contains eosinopoenia (a marker of acute inflammation/sepsis), consolidation and acidaemia; it is an excellent predictor of acute mortality, but not medium-term and long-term outcomes in those who survive to discharge.
The lack of novel predictors, such as cardiac biomarkers, neural respiratory drive,36 4 m gait speed37 and quadriceps size by ultrasound38 may be seen as a limitation. The inclusion of too many indices in model development risks overfitting and loss of performance in the validation cohorts.20 Furthermore, the inclusion of indices that are not routinely collected may introduce bias as missing data are large,39 ,40 and any association may be due to case selection only. For example, in our study only 17% of patients had troponin tests performed, and levels were not related to outcome on univariate analysis. We were unable to capture psychological well-being, social support networks and treatment concordance. These variables are complex to measure and may reduce the usability of a tool.
Previous studies have shown a relationship between anxiety and depression and readmission,41 ,42 although this was not seen in our derivation cohort (based on a preadmission clinical diagnosis). It is possible that an alternative approach to assessing anxiety, such as measurement of the Hospital Anxiety and Depression score at the point of discharge, may add predictive information.
FEV1 is associated with exacerbations and hospital admissions in patients with stable COPD and guides treatment,16 ,43 ,44 but in our derivation cohort it was not an independent predictor of the primary outcome. The better performance of FEV1 in the derivation cohorts of tools such as ADO, BODEX, CODEX and DOSE probably reflects differences in population and measured outcome.
PEARL was superior to ADO, updated ADO, BODEX, CODEX, DOSE and LACE. The CODEX study was developed in a large number of hospitals, the population is clearly described and model performance is appropriately assessed. In its derivation study, it was superior to ADO and BODEX for 90-day readmission or death, although this comparison favours CODEX.4 In our study, it was the second best performing tool. LACE was developed in unselected patients, rather than those with AECOPD,9 for 30-day (and not 90-day) outcome. Of importance, PEARL was superior at both time-points. The LACE score was selected for comparison as it was derived in a well-conducted study, demonstrated better discrimination than other generic tools and is used in some hospitals. The requirement to score the full Charlson Comorbidity Index limits the bedside application of both CODEX and LACE.
A number of studies have shown positive outcomes from interventions aimed to reduce readmission.45–47 There is room for further research to improve outcome, particularly in COPD.48 ,49 The lowest risk group (PEARL 0–1) comprise almost a third of the population. Such risk stratification can inform research by excluding low-risk groups (of importance, the risk of death alone was only 2.5% in the low risk, PEARL 0–1 group), or by using randomisation techniques that include stratification or minimisation by risk group.
In current practice, clinician judgement is used to identify and target resources towards patients with a high readmission risk, although this judgement is known to be poor.3 The PEARL score offers robust and consistent prediction of 90-day readmission or death, and is superior to alternative tools. PEARL may aid clinical decision-making and resource allocation, although quantification of the impact of PEARL in terms of cost and patient outcomes requires further research.
We are grateful to Victoria Ferguson for her input with data management. We are thankful for the hard work of the research and clinical teams in collecting data, including Stephen Gundry, Maureen Armstrong, Peter Fenwick, Liz Norman and her team, Nicola Bateman, June Battram, Angela Scott-Johnson, Valerie O'Dwyer, Karen Heslop-Marshall and her team. We thank Ross C Palmer for the PEARL acronym. Valuable statistical advice was provided by Colin Muirhead. We thank our expert patients, especially Therese Gibson, for their support and input with planned implementation research trials and clinical services.
Contributors SCB designed and obtained funding for the study. CE, GJG and JS contributed to trial design. SCB, MW, RNH, RH and SCS were primary investigators at each site. CE collated the data, performed the statistical analysis and drafted the manuscript. Data analysis and interpretation was supported by SCB, GJG, JS and NS. The manuscript was redrafted by SCB, GJG, JS and CE. All authors were involved with data acquisition, contributed to the interpretation of results and reviewed and approved the final manuscript.
Funding This trial was funded by the Department of Health, Breathe North appeal and Novartis Pharmaceuticals UK, and supported by the NIHR Clinical Research Network. The salary of CE was funded by the Northumbria NHS Foundation Trust Teaching and Research Fellowship programme.
Competing interests None declared.
Ethics approval The study was approved by NRES Committee North East, UK (REC reference: 12/NE/0379).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Any individuals or parties interested in accessing out data should contact SCB.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.