Article Text
PDF
Abstract
Background: Community-acquired pneumonia (CAP) is a leading infectious cause of death throughout the world, including Hong Kong.
Aim: To compare the ability of three validated prediction rules for CAP to predict mortality in Hong Kong: the 20 variable Pneumonia Severity Index (PSI), the 6-point CURB65 scale adopted by the British Thoracic Society and the simpler CRB65.
Methods: A prospective observational study of 1016 consecutive inpatients with CAP (583 men, mean (SD) age 72 (17) years) was performed in a university hospital in the New Territories of Hong Kong in 2004. The patients were classified into three risk groups (low, intermediate and high) according to each rule. The ability of the three rules to predict 30 day mortality was compared.
Results: The overall mortality and intensive care unit (ICU) admission rates were 8.6% and 4.0%, respectively. PSI, CURB65 and CRB65 performed similarly, and the areas under the receiver operating characteristic (ROC) curve were 0.736 (95% CI 0.687 to 0.736), 0.733 (95% CI 0.679 to 0.787) and 0.694 (95% CI 0.634 to 0.753), respectively. All three rules had high negative predictive values but relatively low positive predictive values at all cut-off points. Larger proportions of patients were identified as low risk by PSI (47.2%) and CURB65 (43.3%) than by CRB65 (12.6%).
Conclusion: All three predictive rules have a similar performance in predicting the severity of CAP, but CURB65 is more suitable than the other two for use in the emergency department because of its simplicity of application and ability to identify low-risk patients.
- CAP, community-acquired pneumonia
- ICU, intensive care unit
- LOS, length of stay
- PSI, Pneumonia Severity Index
- ROC curve, receiver operating characteristic curve
Statistics from Altmetric.com
- CAP, community-acquired pneumonia
- ICU, intensive care unit
- LOS, length of stay
- PSI, Pneumonia Severity Index
- ROC curve, receiver operating characteristic curve
Community-acquired pneumonia (CAP) is a leading infectious cause of death throughout the world1–,3 including Hong Kong,4 and a major drain on healthcare resources.5 The estimated annual costs for treating pneumonia in the USA and UK are US$8.4 billion6 and £441 million,7 respectively, with most of these costs being used for inpatient care. In Hong Kong the problem may be even worse since the outbreak of severe acute respiratory syndrome (SARS),8 with some emergency physicians adopting a conservative approach to the management of pneumonia by indiscriminately admitting nearly all patients with CAP into hospital.
Several international organisations have developed guidelines7,9–,11 or scoring systems12–,14 in an attempt to stratify CAP according to risk severity; the aim is to enhance the appropriateness of admission and to lower unnecessary admission rates. The Pneumonia Severity Index (PSI)12 is a widely used predictive method in North America. It is a two-step scoring system using 20 variables and was developed for identifying low-risk patients or potential candidates for outpatient care. CURB13,14 is a modified version of the British Thoracic Society (BTS) assessment tool which relies on four parameters for scoring—namely, mental confusion, blood urea level, respiratory rate and diastolic blood pressure. In 2003 Lim et al14 added age ⩾65 years as a fifth prognostic variable to the CURB scoring system and turned it into a 6-point scoring scale (0–5) known as CURB65 which was adopted by the BTS as the new severity assessment strategy for CAP in 2004.15 A simpler model, the CRB65, without the requirement for laboratory investigations (urea), could be a useful tool outside hospital as it is based only on clinical parameters for scoring (confusion, respiratory rate, blood pressure, and age) and has comparable results to CURB65.14
Some centres have validated these guidelines in their own context,16–,18 but others have reported that as many as 27% of patients with PSI class I–III (low risk) require admission to the ICU19 and 40% of patients in low-risk groups were admitted to hospital.20 These rules have never been tested in Hong Kong, despite the high incidence of CAP and despite recent major outbreaks of atypical pneumonia such as SARS.
The aim of this study was to compare the effectiveness of these three validated prediction rules (PSI, CURB65 and CRB65) to predict mortality and to evaluate their potential application as a guide for admission or discharge from the emergency department. We investigated whether the null hypothesis that there is no difference between the risk stratification groups of the three rules for predicting 30 day mortality on admission can be rejected.
METHODS
Study design and patients
Data were collected prospectively from consecutive patients admitted to hospital through the emergency department with a provisional diagnosis of CAP between 1 January 2004 and 31 December 2004. Ethical approval was obtained from the Institutional Review Boards of the Hospital Authority of Hong Kong and of the Prince of Wales Hospital to conduct a prospective study in patients with CAP.
Setting
The study was conducted in the emergency department of the Prince of Wales Hospital, the main teaching hospital of the Faculty of Medicine of the Chinese University of Hong Kong. The hospital has over 1200 beds, a workforce of 3400 healthcare staff and serves a population of 1.5 million people in the New Territories of Hong Kong. The emergency department sees 160 000 new patients per annum and admits 24% to hospital. Approximately 50 patients are admitted to internal medicine wards each day, of which 20% are admitted with suspected infective causes.
Definitions and CAP scoring systems
CAP was defined in our study as an acute infection of the pulmonary parenchyma that was associated with symptoms of acute infection, accompanied by the presence of an acute infiltrate on a chest radiograph in a patient not hospitalised for more than 14 days before onset of symptoms.10 All patients were assessed by a specialist emergency physician before admission and by a specialist in respiratory medicine or infectious diseases before discharge. The final diagnosis was made by a respiratory physician and was based on the clinical, radiological and laboratory results. Patients with CAP were stratified into low, intermediate and high-risk groups according to the PSI, CURB65 and CRB65 scoring systems.12,14 The rules were then compared for their ability to predict 30 day all-cause mortality.
Inclusion and exclusion criteria
All patients aged more than 17 years, admitted to hospital and who fulfilled the definition of CAP were included in the study. Exclusion criteria were patients with severe immunosuppression, defined as HIV infection, neutropenia <1×109/l, on long-term immunosuppressants or steroids, or solid organ transplant recipients. Patients with a final diagnosis of pulmonary tuberculosis were excluded because of the different disease progress, management and prognosis. The period for this study did not include the SARS outbreak of 2003, and there were no isolated cases in Hong Kong during 2004. We also excluded patients who had been in hospital within the previous 14 days and those with a diagnosis other than CAP after admission.
Radiographic evaluation
All patients had a frontal chest radiograph in the emergency department. Images were assessed using a picture archiving and communication system (PACS), viewer workstation with a 2048×2048 pixel monitor (Magicview Version VA22E; Siemens, Munich, Germany). The images were reviewed by senior radiologists working in pairs and interpretation was by consensus. Radiologists reviewed the x rays of all patients and were blind to the clinical information except that they were aware that this was a study aiming to include patients with CAP.
Data collection
All data were recorded according to a standard questionnaire by a trained research nurse. Data collected at admission to the emergency department included age, gender, admission from home or a nursing home, coexisting illness, symptoms and clinical parameters: blood pressure, pulse rate, respiratory rate, percutaneous oxygen saturation, tympanic temperature and mental confusion (defined in our study as Glasgow Coma Scale ⩽14). Additional data collected for all patients either in the emergency department or immediately after admission included laboratory results (complete blood count, arterial blood gas, glucose, electrolytes and urea), radiographic findings reported by the radiologists and outcome variables (the requirement for ICU admission or mechanical ventilation, length of stay (LOS) in hospital and all-cause 30 day mortality).
Outcome measures
The primary outcome measure was 30 day all-cause mortality. Secondary outcome measures included ICU admission rate, LOS in hospital and ICU-free days (defined in our study as days alive between day 1 and day 30 which were spent outside the ICU).
Statistical analysis
All data were analysed using SPSS Version 13.0 for Windows and Statview for Windows (SAS Institute Version 5.0). Descriptive statistics of demographic and clinical variables included frequencies, percentage, means and standard deviation (SD). One-way ANOVA and Kruskal-Wallis tests were used for comparisons of normal and skewed continuous variables of more than two groups, respectively. Sensitivity, specificity, positive predictive value, negative predictive value and the areas beneath the receiver operating characteristic (ROC) curves for predicting 30 day mortality in each predictive rule were compared. ROC curves were generated first for the population as a whole and then with nursing home residents excluded. For all analyses a two tailed p value of <0.05 was considered statistically significant.
RESULTS
Of 1648 consecutive patients admitted with a provisional diagnosis of CAP, 1016 (62%) were included in the study; 632 patients (38%) were excluded either because they had exclusion criteria or a non-CAP diagnosis. The baseline characteristics of the 1016 patients included in the study are shown in table 1⇓: 583 (57.4%) were male and the mean (SD) age was 72 (17) years, 789 (77.7%) were aged >65 years and 247 (24.3%) were nursing home residents. All patients were treated with empirical antibiotics according to the standard hospital guideline of the Hospital Authority of Hong Kong.21
Baseline characteristics and outcome measures of patients with pneumonia (N = 1016)
Outcome
Eighty-seven patients (8.6%) died within 30 days of admission, 41 (4%) needed ICU care and 25 (2.5%) needed mechanical ventilation. The mean (SD) LOS in hospital was 9.3 (8.5) days.
Comparisons of mortality and ICU admission rate
Table 2⇓ shows the patient distribution and 30 day mortality in each risk score of the predictive rules. The different PSI risk classes and CURB65/CRB65 scores were categorised into low, intermediate and high-risk groups (table 3⇓) according to their original study methodology.7,13 All three predictive rules showed the same trend of increasing mortality with worsening risk groups (p<0.001). The mortality rate of low-risk groups was 2.9% in PSI, 3.0% in CURB65 and 2.3% in CRB65. PSI and CURB65 classified a significantly larger proportion of patients (47.2% and 43.3%, respectively) as low risk than CRB65 (12.6%).
Distribution of patients and 30 day mortality in each risk class of predictive rules*
Comparison of different outcome measures of predictive rules*
ICU admission rates also increased with the risk levels of each rule, but were only statistically significant in CURB65 and CRB65 (p = 0.02 and p = 0.01, respectively). The ICU admission rate of low-risk groups was 2.7% in PSI, 2.3% in CURB65 and 3.9% in CRB65. The mortality rate of high-risk patients was 22.1% in PSI, 19.5% in CURB65 and 24.8% in CRB65. The ICU admission rate was 9.5% in the high-risk group of CRB65, which was higher than PSI and CURB65 (6.6% and 6.5%, respectively). The calculated sensitivity and specificity for the high-risk group of each rule in identifying ICU admission were 29.3% and 82.7% for the PSI, 41.5% and 75.0% for CURB65 and 24.4% and 90.3% for CRB65. A separate analysis using a modified American Thoracic Society (ATS) rule22 was performed in this cohort. Four hundred and thirty-six patients (42.9%) were classified as having severe CAP and 8.5% of them needed intensive care. The sensitivity and specificity for identifying patients needing intensive care were 90.2% and 59.1%, respectively.
Comparisons of LOS in hospital and ICU-free days
The in-hospital LOS increased with increasing risk group, while the number of ICU-free days decreased with increasing severity (table 3⇑). The differences in LOS and ICU-free days in different risk groups were all statistically significant.
Comparison of predictive accuracy
Table 4⇓ shows the sensitivity, specificity, positive and negative predictive values for 30 day mortality at different cut-off points for each scoring system. PSI had higher sensitivities and lower specificities than CURB65/CRB65 at all different risk levels. The sensitivities of low risk cut-off points were 83.9% in PSI, 85.1% in CURB65 and 96.6% in CRB65. All three prediction rules had high negative predictive values but low positive predictive values at all cut-off points.
Sensitivity, specificity, positive and negative predictive values of 30 day mortality of the different predictive rules*
The ROC curves for 30 day mortality for each prediction scoring method are shown in fig 1⇓. There was no significant difference in the area under the ROC curves for each of the three scoring methods. When nursing home residents were excluded, the area under the ROC curves, which are not shown in fig 1⇓, for the remaining patients were 0.728 (95% CI 0.662 to 0.793), 0.713 (95% CI 0.639 to 0.788) and 0.654 (95% CI 0.572 to 0.736) for PSI, CURB65 and CRB65, respectively.
Receiver operating characteristic (ROC) curves of different predictive rules in 30 day mortality.
DISCUSSION
Prediction rules may be useful adjuncts for clinical decision making. This study shows no clinical difference in performance between PSI, CURB65 and CRB65 for a large Hong Kong population.
More than three-quarters of the patients in our study were over 65 years old, reflecting the fact that most patients with CAP requiring hospital management are elderly. The management of this group is often challenging: they are more likely to have atypical clinical presentations, a different distribution of aetiological agents, coexisting illness, poor social support and a higher mortality rate.23 Emergency physicians in Hong Kong tend to adopt a conservative approach, admitting patients from this group into hospital without regard for their disease severity, especially in the absence of a good clinical guideline. This leads to many potentially unnecessary hospital admissions which not only strains hospital resources but also exposes some patients to increased risks of nosocomial infection.1
The mortality rate in our study is comparable to those reported worldwide.7,11 A certain proportion of admitted patients could be considered for outpatient care, and a good predictive rule is needed to help frontline emergency department staff. Similar to the findings in other studies, all three prediction rules had high negative predictive values but low positive predictive values for 30 day mortality at all cut-off points and are therefore more useful in ruling out serious illness.13,14,16
The rules may be used to identify low-risk patients and guide emergency department discharge policies. The difference in mortality between low-risk groups is small, but more than 40% of the patients with CAP were identified as low risk by PSI and CURB65, meaning that at least 40% of the admitted patients could potentially have been considered for outpatient management using PSI and CURB65. However, only 12.6% of patients were identified as low risk by CRB65. PSI and CURB65 are therefore more useful than CRB65 for identifying potential patients with CAP for potential outpatient management. CURB65 is more practical than PSI if simplicity is taken into consideration. For identifying high-risk patients, all three rules have a low positive predictive value which makes them less useful in guiding decision making for inpatient management.
None of the three clinical decision rules appeared to be useful for identifying patients requiring ICU care because of their low sensitivities. When the most recent parameters for the ATS rule were applied to our study population, the sensitivity was higher (90% vs 78%) but specificity lower (59.1% vs 94%) than the original study.22 This suggests that the ATS rule, when applied in our setting, may be more useful as a “rule out” than a “rule in” tool. No prospective studies have clearly related disease severity with ICU admission rates. Generally, patients of higher risk classes have higher rates of ICU admission. However, unlike 30 day mortality, the association between them is not well defined. In fact, the ICU admission rate in our study was low compared with previous work17 (4% vs 16.7%) despite similar mortality rates in both studies. It is clear that the criteria for ICU admission vary from country to country and from hospital to hospital, and disease severity is not the only factor to consider. Other factors such as disease prognosis, pre-morbid status, age of patient, and the availability of ICU resources are all routinely considered by intensive care physicians before admitting a patient to the ICU.
Our study comprised mainly elderly patients who may have more coexisting illness, poorer quality of life and poorer prognoses than younger patients; these patients may have a lower chance of receiving intensive care than the general population. Indeed, the 30 day mortality rate in our study was higher than the ICU admission rate (8.6% vs 4%), meaning that at least some patients with severe disease died without prior ICU care. Prediction rules are therefore not useful in predicting ICU admission, especially among elderly patients, although they give an indication of disease severity.
The relationship between ICU-free days and disease severity may be better than the relationship between ICU days or LOS in hospital and disease severity as patients with more severe illness may die earlier and have a short LOS in hospital. The in-hospital LOS and ICU-free days depend on the time needed to reach clinical stability, which is closely related to the severity of the illness.24 High-risk patients need more time to reach clinical stability, and hence have longer stays in hospital and fewer ICU-free days.
Although the characteristics of nursing home residents with pneumonia may be different from patients with CAP,25 in practice emergency physicians are unlikely to differentiate between these two groups. Assessment of prognosis will depend upon physiological factors rather than the differentiation of nursing home versus non-nursing home residents. Although nosocomial pneumonia and CAP may have different bacteriological aetiologies, when our database was analysed with and without their inclusion, there was no significant difference in predictive ability or risk stratification. Therefore, for the purpose of this study, we included these residents in our analysis.
The strength of this study lies in its prospective design, large sample size, the completeness of data collection and a tertiary teaching hospital setting with a wide catchment area. All radiographs were reported by senior radiologists who were blinded to the disease conditions of the patients, giving rise to less potential bias in radiological interpretation. A further strength is that current clinical policy in our institution is to obtain arterial blood gas measurements from all patients admitted with CAP irrespective of their oxygen saturation level.
One limitation of our study is the sample selection—namely, all enrolled patients were inpatients admitted through the emergency department with a diagnosis of CAP. These patients were, on average, more severely ill and more advanced in age. PSI, CURB65, CRB65 and most other predictive rules were not originally designed for use in elderly patients, and this may account for the overall lower accuracy of the three predictive rules in our study compared with other validation studies.16,18 The “confusion” variable used in our study was not the same as in the definition of CURB65. The definition of confusion for CURB65 was an Abbreviated Mental Test Score of ⩽8 or new disorientation to person, place and time.13 However, for our study, we defined confusion as a Glasgow Coma Scale score of ⩽14; many different dialects are used in Hong Kong, especially among the elderly which makes it impossible to use the Abbreviated Mental Test Score to screen our participants.
All predictive rules serve only as a guide to clinical management, and severity of illness is not the only factor which should be considered when deciding on whether or not to admit a patient. Social and home circumstances must be considered fully in reaching a clinical management decision, especially with elderly patients. Physicians should always exercise clinical judgment and common sense in making these sometimes difficult decisions.
In conclusion, no significant differences were found between PSI, CURB65 and CRB65 in predicting 30 day mortality. CURB65 may be more useful than PSI and CRB65 in a busy emergency department because of its simplicity in application and its ability to identify a reasonable proportion of low-risk patients for potential outpatient care.
REFERENCES
Footnotes
Published Online First 22 November 2006
Funding and support: The authors received no outside funding or support.
Competing interests: None declared.