Introduction

Unplanned hospital readmissions within 30 days are estimated to cost the health service in England over £2 billion per year and over $17 billion per year in US Medicare expenditure.1 2 Reduction strategies targeting readmissions have therefore been a focus for policy makers through quality improvement activities and financial penalties.2 3 Intensive care unit (ICU) survivors are known to experience increased mortality, use more acute hospital resource and reduced quality of life in the years following hospital discharge.4–8 This increased morbidity has been termed the ‘post-intensive care syndrome’ and may leave ICU survivors and their caregivers with less resilience to new acute stressors as well as persisting problems related to the acute critical illness.9

Unplanned hospital readmission is a potentially useful outcome measure in ICU survivor populations. It is easy to measure in linked information systems, is associated with increased costs and may reflect the effectiveness of rehabilitation interventions, which are increasingly considered a standard of care following critical illness10 despite recent conflicting trial evidence.11 However, the validity of unplanned readmission as an outcome measure requires an understanding of contributing factors, especially those that are potentially modifiable by intervention within survivor populations.

Although statistical models have been developed to predict readmission risk for many hospitalised patient groups,12–14 none have specifically assessed ICU populations with risk factors related to the critical illness episode, for example, organ dysfunction. An ICU-specific model could potentially identify survivors at high risk and might enable screening of survivors before hospital discharge in whom to target novel risk reduction strategies.

As part of a mixed methods programme exploring drivers of unplanned readmission following critical illness,15 we undertook a national cohort study to quantify the proportion of ICU survivors experiencing readmission within 90 days of discharge and identify risk factors for 90-day readmission. We also aimed to develop a risk prediction model and assess its performance as a screening tool to identify ICU survivors at highest readmission risk.

Methods

Results

In total, 55 975 patients were admitted to ICUs and discharged alive (online supplementary figure 1). Median age was 60 years (IQR 45–71), and patients living in the most deprived regions were over-represented (49.2% resident in two most deprived quintiles, 40% in general population) (table 1; online supplementary table 1). Pre-existing illness and morbidity was prevalent: 31.3% had an unplanned admission during the previous year, while 56.4% had at least one comorbidity. Previous alcohol-related (10.8%) and drug-related morbidity (7.0%) were prevalent (online supplementary table 1). The most common admission diagnosis was pneumonia (8.4%).

Of 55 975 patients, 13 471 (24.1%, 95% CI 23.7% to 24.4%) experienced unplanned 90-day readmission (figure 1). A further 712 (1.3%, 95% CI 1.2% to 1.4%) died without being admitted. A total of 14 183 patients (25.3%, 95% CI 25.0% to 25.7%) experienced 90-day readmission/death). An additional 1015 (1.8%, 95% CI 1.7% to 1.9%) died within 90 days, but these deaths occurred after an unplanned readmission.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Cumulative incidence within 90 days of discharge from index hospital stay of (A) unplanned hospital admission and (B) unplanned hospital admission or death.

For full version of baseline characteristics table and missing data, see online supplementary table 1. Unplanned readmission proportions for continuous variables are presented in online supplementary table 2.

Predictors of 90-day unplanned hospital readmission

Patient demographics

In univariable analyses, all demographic factors other than sex had statistically significant associations with readmission risk (older age, social deprivation and remoteness of residence (online supplementary table 2)). As a combined group, the c-index was 0.54 (95% CI 0.54 to 0.55), indicating weak discriminant ability (figure 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

(A) Observed risk of 90-day unplanned hospital readmission by deciles of predicted risk and (B) receiver operator characteristics for three groups: patient demographics, indices of pre-existing patient health and indices of critical illness severity. Within each panel, each point represents 10% of the cohort grouped by their predicted risk of 90-day readmission derived from the group of characteristics labelled by the panel axis label. The observed risk for ‘Demographics’ variables ranges from 18.2% in the lowest predicted risk decile to 29.4% in the highest. The observed risk for the ‘pre-existing health indices’ group of variables ranges from 15.3% in the lowest predicted risk decile to 46.9% in the highest. The gradient of the line is therefore steeper. The steeper positive gradient observed for ‘Pre-existing health indices’ compared with ‘Demographics’ indicates that there is a greater increase in observed readmission risk for each increment in decile of predicted risk for the group of characteristics, and therefore this group is a better predictor of readmission across the range of predicted risk.

Indices of pre-existing patient health

Prior health resource use and comorbidities demonstrated better discrimination for readmission risk. The number of previous unplanned inpatient admissions was associated with readmission rates from 19.5% (95% CI 19.1% to 19.9%) (zero admissions) to 70.2% (95% CI 66.6% to 73.6%) (six or more) (online supplementary table 2; c-index 0.60). Pre-existing comorbidities demonstrated moderate discrimination overall (c-index 0.60). In those experiencing an unplanned 90-day readmission, 68.7% (95% CI 67.9% to 69.5%) had at least one comorbidity. All individual comorbidities were associated with increased risk (online supplementary figure 3A, greatest risk: renal disease, moderate/severe liver disease, diabetes with complications with >40% risk). As a combined group, indices of pre-existing health and resource use demonstrated moderate discrimination (c-index 0.63, 95% CI 0.63 to 0.64), which was the highest compared with the other two groups (figure 2, χ²=389, 2df, P<0.001). This was reflected in improvement in the classification of patients not experiencing a readmission of 31.5% and 10.7% in comparison with demographics and critical illness severity indices, respectively, at the expense of worse reclassification of patients experiencing a readmission (−18.2% and −6.1%, respectively) (online supplementary table 3A, B).

Indices of critical illness severity

Overall, diagnostic category (c-index 0.57) and APACHE II score (c-index 0.55) were weak discriminators (online supplementary table 2). Some specific diagnostic categories were associated with high readmission risk (variceal bleed (45.8%; 95% CI 41.3% to 50.4%) and pancreatitis (40.0%; 95% CI 36.1% to 44.1%)). Organ support variables were weak discriminators of 90-day readmission (c-index range 0.51–0.52). Similarly, length of post-ICU hospital stay and overall length of hospital stay were weak discriminators (c-index 0.52–0.56). As a combined group, the c-index for indices of critical illness severity was 0.60 (95% CI 0.60 to 0.61) (figure 2).

Multivariable analyses

In multivariable analyses, number of previous unplanned admissions was strongly associated with risk of 90-day readmission, with a predicted absolute risk increase from 20.3% (95% CI 19.9% to 20.8%) in those with no previous readmissions to 61.1% (95% CI 57.7% to 65.5%) in those with six or more (OR 6.19, 95% CI 5.12 to 7.49) (online supplementary table 4). Readmission risk increased with comorbidity count, from 19.5% (no comorbidities; 95% CI 18.8% to 20.1%) to 34.5% (5 or more; 95% CI 30.6% to 38.5%). Replacing comorbidity count with individual comorbidities revealed seven individual comorbidities no longer retained statistical significance (online supplementary figure 3B). Several other factors remained statistically significant, but the gradient of readmission risk across categories was less pronounced; these included age, type of admission to ICU and length of post-ICU hospital stay. Several specific diagnoses were independently associated with predicted risk substantially higher than the population mean, namely oesophageal variceal bleed (33.5%, 95% CI 28.8% to 38.1%) and pancreatitis (38.4%, 95% CI 34.0% to 42.7%). Several factors were not significant predictors in multivariable analysis, including socioeconomic status, Acute Physiology Score (APS) and ICU length of stay.

Risk prediction model

In the derivation cohort (n=33 294, online supplementary table 5), the overall discriminant ability of the model was fair (c-index 0.67, 95% CI 0.66 to 0.67) and overall performance was acceptable (Brier’s score 0.170). In the validation cohort (online supplementary table 6), discriminant ability and overall performance were similar (c-index 0.65, 95% CI 0.64 to 0.66; Brier’s score 0.176; figure 3). Model calibration across the range of predicted risk in the validation cohort was reasonable, although the model slightly underpredicted readmission risk (mean observed risk 25.0%; mean predicted risk 23.6%) (online supplementary figure 4).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Receiver operator characteristics (ROC) curve for model predicting unplanned hospital readmission within 90 days of discharge from index hospital stay in the validation cohort (n=14 273).

Performance of risk prediction model as a screening tool

The model’s performance as a screening tool is illustrated using the validation cohort in table 2 (see online supplementary table 7 for 95% CI). Assigning ≥20% predicted probability of the outcome as the threshold to ‘screen positive’ would lead to the majority (54.2%, 95% CI 53.4% to 55.0%) of patients screening positive with a 30.7% (95% CI 29.2% to 32.3%) false-negative rate and a 49.2% (95% CI 48.2% to 50.1%) false-positive rate. Increasing the threshold to ≥50% improves the probability of identifying a patient who will subsequently experience a readmission but a substantially smaller proportion of the population screen positive (decreasing to 1.6%, 95% CI 1.4% to 1.8%). Online supplementary table 8 illustrates the patient characteristics and outcomes of those who would screen positive compared with those who would screen negative at two probability thresholds (≥20% and ≥50%).

View this table:

• View inline
• View popup

Table 2

Performance of risk prediction model as a screening tool to identify patients at risk of unplanned hospital readmission

Discussion

In a large, complete population study, we have demonstrated that 1 in 4 ICU survivors experience an unplanned readmission within 90 days of hospital discharge. Indices of pre-existing ill health were more strongly predictive of readmission than indices of critical illness severity in the whole cohort, but this was reversed in subgroup analyses of patients with no recorded comorbidity. A risk prediction model derived from multiple data sources had, at best, only moderate discriminant ability. A screening tool derived from this model is unlikely to perform sufficiently well in isolation to identify cases in whom to target high intensity interventions aimed at reducing readmissions among ICU survivors.

Our data indicate unplanned readmission rates among ICU survivors are substantially higher than the general hospital population (30-day readmission 14.7% in ICU survivors vs 7.0% in all hospital inpatients22). Unplanned readmission rates are increasingly used as a quality indicator and target for improvement.23 24 Although many variables had statistically significant associations with readmission risk, almost all had limited discriminant power as individual factors. A key finding was that ICU-related factors such as organ support are not independently associated with readmission risk among survivors. In contrast, pre-existing health factors had the greatest predictive power of all variables. However, in the subgroup of patients with no pre-existing comorbidity, indices of critical illness had greater discriminant power than pre-existing health factors. These findings are consistent with precritical illness chronic health being the dominant factor at a population level in general critical care survivors in determining post-ICU health trajectories, whereas new impairments that follow an ICU admission may be more dominant in subgroups with no comorbidity.25–27

Addressing recovery from critical illness from this perspective has important implications for research, policy and service design given the high prevalence of older patients with comorbidity in critical care populations.4 6 For example, it may explain the lack of effect on clinical outcomes from rehabilitation interventions focused mostly on physical therapy alone7 and also questions the rationale for using outcomes such as longer term hospital costs and health-related quality of life in critical care trials without accounting for preillness health status.

Our model had a similar discriminant ability compared with other published risk prediction scores used in general hospitalised populations (c-indices of studies using retrospective administrative data 0.55–0.72).28 This was despite inclusion of ICU-related/acute factors. These findings may be explained by our datasets not including important factors associated with readmission risk, such as social and organisational factors identified in a systematic review.28 Research in other populations highlights the importance of these factors, which have not previously been well addressed in ICU survivor populations. For example, family stress,29 lack of information 30 and frailty31 could all be important during the early posthospital period. Furthermore, some patients will experience readmissions due to unpredictable factors that would not be present in exhaustively comprehensive datasets. Analysis of PROFILE’s qualitative component, comprising interviews and focus groups with ICU survivors and their carers, may reveal additional insights.15 Our study clearly shows that additional research is needed to understand other factors driving readmission risk in this population to improve the discriminant value of a clinical decision support tool.

We used 90-day unplanned readmission as our primary outcome, whereas 30-day readmission is widely used in other patient groups.12 27 28 32 We believe the longer time period is justified, because the ICU survivorship literature shows a longer period ‘at risk’, both for increased mortality and hospital resource use.4–6 In addition, a sensitivity analysis using 30-day readmission as the outcome was similar to the primary outcome. Furthermore, HRQoL typically starts to plateau in ICU survivors after 3 months,7 31 and this time point is widely used for primary outcome measurement in critical care trials.7 33 Extending the period of interest further risks including readmissions that are less causally related to the critical illness hospitalisation.

Our study has a number of strengths. The database had complete population coverage, included a diverse range of data sources that undergo regular validation and contained a large number of events, resulting in unbiased, precise estimates. We undertook sensitivity analyses to explore the effect of death as a competing risk. We reported our risk prediction model using current best practice and undertook internal validation using a recommended time-based split.21 Other risk prediction scores for readmission have minimised the number of variables to ensure ease of clinical use.12 34 35 We decided a priori to pursue a non-parsimonious approach to model building with the intention of electronic implementation.

There are potential limitations to our study. We were unable to access measurements of preadmission functional status, frailty trajectories or biomarkers relating to inflammation, which have been associated with poorer health outcomes following critical illness.31 32 36 37 Furthermore, we had no data on social care or other non-clinical variables that have been shown to influence readmission risk, for example, polypharmacy,14 or low health literacy.38 These are not routinely considered in critical care recovery pathways.

Our study has a number of methodological limitations. While a time-based split is a robust method of model validation, secular trends in demographics, clinical practice and healthcare organisation can bias model performance. This may mean time-based validation methods perform worse than methods in which derivation and validation cohorts are drawn from the same time period. In addition, using statistical significance to select variables may have resulted in more complex models than, for example, using change in Bayesian information criterion. Furthermore, 12.8% were missing APACHE II scores. This is due to specific APACHE II model exclusions, rather than being ‘missing’, and these values cannot therefore be imputed. This means our model cannot be generalised to those patients excluded from APACHE II scoring.

Our design did not enable an assessment of the proportion of readmissions that might be avoided through interventions. A recent systematic review estimated the median proportion of avoidable readmissions was 27%.39 This research question may be better investigated using qualitative methodology, which was the approach used in a parallel part of our research programme.15 Understanding the modifiable factors that cause readmissions in critical care survivors is essential for designing effective anticipatory interventions. Developing and testing such interventions requires detailed understanding of the factors that may be important.40 While we could only report presence of comorbidity, our study suggests optimising chronic disease management is at least as important as strategies specific to the complications of critical illness.

Our results have important implications for future research and policy. The unplanned readmission rates we report in ICU survivors are similar to those with chronic disease currently targeted with specific discharge pathways and community support.41 42 Although guidelines promote rehabilitation after critical illness,10 43 the most clinically and cost-effective way to deliver these are unknown and evidence-based care pathways do not yet exist, in contrast with other conditions such as myocardial infarction42 and stroke.44 Our data support the need for clear pathways with appropriate support for ICU survivors during care transitions, especially from secondary into primary care.

Conclusion

We have demonstrated that 1 in 4 patients experience an unplanned hospital readmission within 90 days of discharge following an episode of critical illness. Pre-existing illness indices are better predictors of readmission risk than acute illness factors at a whole cohort level. In a subgroup of those with no comorbidity, acute illness factors predominate. Identifying additional patient-centred drivers of readmission may improve risk prediction models. Improving our understanding of patient groups and risk factors that are amenable to intervention could improve the clinical and cost-effectiveness of post-ICU care and rehabilitation.