Background Accurate differentiating diagnosis is essential for choosing treatment for exudative pleural effusions.
Objective To establish the diagnostic accuracy of interleukin 27 for tuberculous pleural effusion (TPE).
Methods First, the concentrations of pleural interleukin 27, interferon-gamma and adenosine deaminase were compared between 51 patients with TPE and 103 with non-TPEs (Beijing cohort), and their diagnostic values were evaluated. These were further verified in another independent population (Wuhan cohort, n=120). In the second part of the study, we performed a meta-analysis.
Results With a cut-off value of 591.4 ng/L in the Beijing cohort, the area under the curve, sensitivity, specificity, positive predictive value and negative predictive value of interleukin 27 to diagnose TPE were 0.983 (95% CI 0.947 to 0.997), 96.1% (86.5% to 99.5%), 99.0% (94.7% to 100%), 98.0 (89.4 to 99.9) and 98.1 (93.3 to 99.8), respectively. Excellent diagnostic accuracy of interleukin 27 was also found in the Wuhan cohort and was further confirmed in the meta-analysis. The diagnostic performance of interleukin 27 was comparable to that of interferon-gamma and was more accurate than that of adenosine deaminase. Since the post-test probability of a negative result was always <0.1%, a negative test was considered to exclude TPE in all tuberculosis prevalence settings.
Conclusions Interleukin 27 can be used to diagnose TPE in a high prevalence setting, and a negative result can also be reliably used to rule out TPE in all prevalence settings.
- adenosine deaminase
- interleukin 27
- tuberculous pleurisy
Statistics from Altmetric.com
What is the key question?
Can tuberculous pleural effusion be distinguished from those pleural effusions with the other aetiologies using interleukin 27 concentrations in pleural fluid?
What is the bottom line?
In this study we show that interleukin 27 is a valuable indicator for ruling in tuberculous pleural effusion in a high prevalence setting, and for ruling out tuberculous pleural effusion in all prevalence settings.
Why read on?
A design including two prospective independent cohorts and a meta-analysis allowed this study to establish the exact diagnostic accuracy of interleukin 27 for tuberculous pleural effusion.
Although it is curable, tuberculosis remains one of the most common causes of pleural effusions globally, especially in the developing countries.1 Tuberculous pleural effusion (TPE) is one of the most common forms of extrapulmonary tuberculosis.2 The definitive diagnosis of TPE depends on the demonstration of Mycobacterium tuberculosis in the sputum, pleural fluid or pleural biopsy specimens.3 A presumptive diagnosis can also be established with reasonable certainty by demonstrating granuloma in the parietal pleura1–3 or elevated concentrations of adenosine deaminase (ADA)4 or interferon-γ (IFN-γ)5 in pleural fluid.
A few years ago, we reported for the first time that interleukin 27 (IL-27)—a cytokine of the IL-12 cytokine family produced by antigen-presenting cells under the stimulation of pathogen-associated molecular patterns binding to their toll-like receptors—can distinguish between TPE and non-TPEs, with a sensitivity of 92.7% and a specificity of 99.1%.6 In another study we indicated that the diagnostic efficiency of IL-27 is better than that of ADA.7 On the other hand, one study from a country with a low tuberculosis burden showed that IL-27 is less efficient than ADA,8 and two studies showed that the product of IL-27 and ADA improves the diagnostic sensitivity of ADA.8 9 We therefore performed these prospective studies, with two independent replication cohorts and a meta-analysis, to establish the precise role of IL-27 for the diagnosis of TPE.
The study was approved by the ethics committees of both participating institutions, and all study participants provided written informed consent. Between January 2014 and June 2015, all consecutive adult patients with pleural effusions admitted to the Department of Respiratory and Critical Care Medicine, Beijing Chaoyang Hospital, Capital Medical University, Beijing (Beijing cohort) were screened for this prospective study (see online supplementary figure 1). One hundred and fifty-four patients with an established definite cause of pleural effusion were included in the study. This group was used to evaluate the diagnostic accuracy of IL-27, ADA and IFN-γ for TPE. This was subsequently validated in an independent cohort of 120 consecutive patients with pleural effusions who were recruited prospectively during the same period from the Department of Respiratory and Critical Care Medicine, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan (Wuhan cohort, online supplementary figure 2). The characteristics of both study cohorts are summarised in table 1, and the aetiological distribution of the pleural effusions is presented in online supplementary table 1).
Supplementary file 1
Supplementary file 2
TPE was diagnosed if Ziehl-Neelsen stains or Lowenstein-Jensen cultures of pleural fluid, sputum or pleural biopsy specimens were positive or if granulomas were present in the parietal pleural biopsy specimens (online supplementary table 2). A diagnosis of malignant pleural effusion was established by demonstration of malignant cells in pleural fluid and/or on pleural biopsy specimens. A parapneumonic pleural effusion was diagnosed as any effusion associated with bacterial pneumonia, lung abscess or bronchiectasis; empyema was also included in this group. The remaining effusions comprised miscellaneous pleural effusions.
The patients recruited to this study were those with definite known causes contributing to their pleural effusions. At the time of sample collection, none of the patients had received any anticancer treatment, antituberculosis therapy, corticosteroids or other non-steroidal anti-inflammatory drugs.
Measurement of IL-27, ADA and IFN-γ
Pleural fluid was collected by diagnostic thoracentesis before the patient had received any treatment and was rapidly transferred to the laboratory with a blood sample obtained simultaneously from the same patient. Pleural fluid and blood samples were centrifuged at 1500 rpm for 10 min at 4°C, and the supernatants were aliquoted and stored at −80°C awaiting measurement of IL-27, ADA and IFN-γ.
The concentrations of IL-27 and IFN-γ in pleural fluid and serum were measured by ELISA kits (eBioscience, Inc, San Diego, California, USA) according to the manufacturer’s protocols. ADA activity was determined using colorimetric method kits (InTec Products, Inc, Xiamen, China) in accordance with the manufacturer’s instructions. The technologists running the assays were blinded to the nature of the samples, and the code was broken to the statisticians after construction of the database.
The sample sizes were calculated based on the assumption of a sensitivity of 92.7% and a prevalence of TPE of about 40% obtained in our previous study.6 With a two-sided α of 5%, we aimed to recruit about 130 patients in both the Beijing and Wuhan cohort to obtain a precision of 7% in the estimation of the sensitivity of pleural IL-27 in the diagnosis of TPE.
The concentrations of IL-27 and the activities of ADA were normally distributed, as determined by the Kolmogorov-Smirnov test, and are presented as means±SEMs. IFN-γ data are presented as medians (25th to 75th centiles) since they were not normally distributed. Differences between two or multiple groups were compared using Student’s t test, Mann–Whitney U test, one-way analysis of various (ANOVA) or Kruskal-Wallis ANOVA on ranks, as appropriate. All tests were two tailed and the p value for multiple testing was adjusted by Bonferroni’s test. Comparisons of data in the pleural effusion and in the corresponding serum were made using a paired t test or Wilcoxon signed-rank test, as appropriate.
Receiver operating characteristic curves were drawn, and areas under the curves (AUCs) were calculated to determine the diagnostic value of the concentrations of each biomarker in the pleural effusion, including sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, positive predictive value and negative predictive value10 11; AUCs were compared using the z statistic with the Hanley and McNeil procedure.12 The optimum cut-off values were defined based on their maximum Youden index (sensitivity+specificity−1). The parameters of diagnostic accuracy are shown together with their 95% CIs. To verify the diagnostic accuracy of IL-27, ADA and IFN-γ in the Wuhan cohort, cases with the concentration of each biomarker complying with the corresponding cut-off value obtained in the Beijing cohort, were considered as positive results. All statistical analyses were performed with SPSS and MedCalc software, and p<0.05 was considered to indicate statistical significance.
Medline using the PubMed interface, Embase and the Web of Science were searched for suitable studies up to 8 November 2016; no early date limit was applied. Search keywords included: ‘IL-27/interleukin 27’, ‘pleurisy/pleural effusion’ and ‘tuberculous/tuberculosis’. Articles were also identified by use of the related-articles function in PubMed. References of articles identified were searched manually. Two reviewers independently judged study eligibility while screening the publications. A study was included in the meta-analysis when it reported the concentrations of IL-27 in pleural fluid. Data retrieved from the studies included participant characteristics, test methods, sensitivity and specificity data, cut-off values, publication year and methodological quality.
Using two statistical software programs (Stata, version 9; Stata Corporation; College Station, Texas, USA; and Meta-DiSc for Windows; XI Cochrane Colloquium; Barcelona, Spain), the pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio and diagnostic OR of IL-27 were computed by standard methods.13 The analysis was based on a summary receiver operating characteristic curve.13 14 The sensitivity and specificity for the single test threshold identified for each study were used to plot a summary receiver operating characteristic curve.14 A random-effects model was used to calculate the average sensitivity, specificity, positive likelihood ratio, negative likelihood ratio and diagnostic OR across the studies.15 16 The Χ2 and Fisher exact tests were used to detect statistically significant heterogeneity across studies. Since publication bias is of concern for meta-analysis of diagnostic studies, we tested for the potential presence of this bias using funnel plots and the Egger test.17
Post-test probability of TPE was estimated by the formula: post-test probability=post-test odds/(post-test odds+1), where post-test odds = (prevalence/[1 – prevalence])×likelihood ratio.
In this study, pleural effusions without a definite known cause were excluded, as they might have been malignant but missed by cytology. A total of 154 patients in the Beijing cohort (online supplementary figure 1) and 120 in the Wuhan cohort (online supplementary figure 2), who had paired pleural fluid and serum samples for determining IL-27, ADA and IFN-γ, were included. No differences in the demographic, cytological and biochemical characteristics of the pleural effusions were found between the Beijing and Wuhan cohorts (table 1).
Concentrations of IL-27, ADA and IFN-γ in pleural effusions
As expected and consistent with our previous findings,6 7 we once again observed that the concentrations of IL-27, ADA and IFN-γ in TPE were significantly higher than in patients in the non-TPE group (Bonferroni’s test, all p<0.001), and that there were no differences among malignant, parapneumonic and miscellaneous effusions (Bonferroni’s test, all p>0.05) (table 2 and figure 1A, C, E). We also noted that in the patients with TPE concentrations of IL-27, ADA and IFN-γ were all higher in pleural fluid than serum (all p<0.001). In addition, the serum concentrations of IL-27 in patients with TPE did not differ from those in the three non-TPE groups (Bonferroni’s test, all p>0.05).
Diagnostic values of IL-27, ADA and IFN-γ in the Beijing cohort
With a cut-off value of 591.4 ng/L in the Beijing cohort, the AUC of IL-27 to differentiate TPE from non-TPEs was 0.983 (95% CI, 0.947 to 0.997; p<0.001) (figure 1B and table 3). Also as shown in table 3, the sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, positive predictive value and negative predictive value of IL-27 were 96.1%, 99.0%, 99.0, 0, 98.0 and 98.1, respectively.
The parameters of diagnostic accuracy of ADA and IFN-γ are also presented in figure 1 and table 3. ADA and IFN-γ had AUCs of 0.900 (0.842 to 0.943) (figure 1D) and 0.973 (0.933 to 0.992) (figure 1F), respectively. A comparison of the receiver operating characteristic curves showed that the AUC of ADA was much lower than that of IL-27, with a difference between the areas of 0.082 (95% CI 0.018 to 0.147; z=2.514, p=0.012). The AUC of ADA was also lower than that of IFN-γ (0.072 (0.010 to 0.135); z=2.287, p=0.022). There was no significant difference in the AUCs between IL-27 and IFN-γ (0.010 (‒0.020 to 0.040); z=0.643, p=0.521).
It has been reported that the product of IL-27 and ADA is more accurate than ADA for diagnosing TPE,8 9 but this was not shown in our study. Although a perfect specificity of the product of IL-27 and ADA could be seen (100.0%), it was at a cost of the reduced sensitivity (90.2%). Z-statistical analysis further confirmed that the AUC of the product of IL-27 and ADA was not higher than that of IL-27 (0.003 (‒0.025 to 0.031); z=0.227, p=0.820).
Diagnostic values of IL-27, ADA and IFN-γ in the Wuhan cohort
We performed separately another prospective blinded validation study (Wuhan cohort) using the cut-off values of IL-27, ADA and IFN-γ obtained from Beijing cohort. Excellent discrimination was found with IL-27 between 44 patients with TPE and 76 controls in the Wuhan cohort (table 4 and online supplementary figure 3). Similar to the findings from the Beijing cohort, z-statistical analysis showed that the AUC of IL-27 was not higher than that of IFN-γ (0.029 (‒0.027 to 0.085); z=1.024, p=0.306), but was significantly higher than that of ADA (0.105 (0.030 to 0.180]; z=2.728, p=0.006). In addition, the AUC of IFN-γ was also higher than that of ADA (0.075 (0.006 to 0.144); z=2.141, p=0.032).
Overall diagnostic accuracy of IL-27 from meta-analysis
After independent review, we found that pleural concentrations of IL-27 had been reported in five publications.6–9 18 Our study was conducted separately in two different populations, and thus we treated these research data as two independent studies. Thus, seven studies were considered eligible for inclusion in the meta-analysis (online supplementary figure 4), and the characteristics of these studies are shown in online supplementary table 3. The average samples size of the seven pleural IL-27 studies included was 165 (range 76–431); the subjects included 323 patients with TPE and 834 with non-TPEs. In two publications8 18 and our current two studies, the diagnosis of TPE was made by bacteriological or histological studies or both (gold standard); in the remaining three publications,6 7 9 the majority of patients with TPE were diagnosed based on bacteriological or histological results, and a minority were diagnosed based on clinical course, including clinical presentation, pleural fluid analysis, radiology and the responsiveness to antituberculosis chemotherapy.
We assessed methodological quality using guidelines published by the QUADAS (quality assessment for studies of diagnostic accuracy, maximum score 14) tool19 (ie, appraisal by use of empirical evidence, expert opinion and formal consensus to assess the quality of primary studies of diagnostic accuracy) and the STARD (standards for reporting diagnostic accuracy, maximum score 25) initiative20 (ie, guidelines that aim to improve the quality of reporting in diagnostic studies). As shown in online supplementary table 3, the quality of study design and reporting diagnostic accuracy of most studies were good, as all studies had high QUADAS scores (≥10) and only one study18 had a low STARD score (<15). In addition, the funnel plots for publication bias showed some asymmetry (online supplementary figure 5), and the Egger test was significant for IL-27 (p=0.011), indicating that there was a potential for publication bias.
Figure 2A shows the forest plot of sensitivity and specificity for seven pleural IL-27 assays in the diagnosis of TPE. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio and diagnostic OR was 93.8% (95% CI 90.6% to 96.2%), 91.7% (89.6% to 93.5%), 29.87 (7.47 to 119.35), 0.08 (0.05 to 0.12) and 427.86 (101.26 to 1807.80), respectively. Figure 2B shows that the summary receiver operating characteristic curve of IL-27 was positioned very near the desirable upper left corner, and that the maximum joint sensitivity and specificity was 0.930 (SEM 0.026); while the AUC was 0.976 (SEM 0.015).
When the pre-test prevalence of TPE is 10% (a hypothetical patient at low-risk) and when the IL-27 result was positive, the estimated post-test probability was 76.7% (table 5); whereas a negative test result almost certainly excluded TPE (post-test probability, 0.009%). For a patient with a 25% or 50% pre-test risk, a positive test result increased the probability to 90.8% or 96.8%, respectively; while the absence of a high IL-27 concentration reduced the disease probability to 0.025% or 0.074%, respectively.
A definite diagnosis of TPE is confirmed only when Mycobacterium tuberculosis is demonstrated in sputum or pleural specimens, or when epithelioid cell granulomas and/or caseating granulomas are found in pleural biopsy specimens.1 2 However, sensitivities of microscopic examination (0–1%)21–23 and culture (24–58%)22–25 of pleural fluid for diagnosing TPE are not high owing to the low load of Mycobacterium tuberculosis. Therefore, differential diagnosis of TPE sometimes requires more invasive methods, such as medical thoracoscopy, when one or more thoracenteses fail to establish a definite diagnosis. Almost all cases of TPE can be diagnosed by medical thoracoscopy26; however, not everyone is suitable for, or willing to undergo, medical thoracoscopy.
Where diagnosis of TPE is difficult a number of soluble biomarkers in pleural fluid have been evaluated extensively for this purpose.27 Two previous meta-analyses showed that nucleic acid amplification tests and T cell IFN-γ release assays cannot replace conventional tests in the diagnosis of TPE.28 29 Another two meta-analyses showed that pleural ADA and IFN-γ are two valuable biomarkers for such a diagnostic purpose.4 5 Our study confirmed our previous findings6 7 and provided stronger evidence to support the use of IL-27 as an excellent diagnostic biomarker for discriminating between TPEs and non-TPEs, with high sensitivity and high specificity of 96.1% and 99.0%, respectively (Beijing cohort).
Our data from the Beijing cohort also showed that a positive likelihood ratio value of 99.0 with IL-27 suggested that the patients with TPE have a 99-fold higher chance of being IL-27-positive than patients without the disease, which is sufficiently high for diagnosis. This may help to avoid unnecessary invasive diagnostic investigations in a number of patients. Moreover, a negative likelihood ratio value of 0 suggests that if the IL-27 result is negative, there is no probability that this patient has TPE. The high positive predictive value (98.0) and high negative predictive value (98.1) of IL-27 found in this study further indicate that both the false-negative rate and false-positive rates are low. One important strength of this study was that the excellent diagnostic performance of IL-27 found in the Beijing study was verified prospectively in the Wuhan study and further confirmed in the meta-analysis.
Despite the relatively high overall diagnostic accuracy of ADA reported in a previous meta-analysis,4 intense criticism of its use for diagnosis has been made as its performance is highly dependent on the prevalence of TPE in the studied populations.30–32 One study from Spain showed that IL-27 is less efficient than ADA in the diagnosis of TPE.8 In contrast, by a head-to-head comparison of the diagnostic measurements, we found in both the Beijing and Wuhan cohorts that IL-27 is significantly better than ADA for discriminating between TPEs and non-TPEs. In addition, it has been shown that the product of IL-27 and ADA is more accurate than ADA alone in European countries.8 9 We failed to observe similar results, which is unsurprising as IL-27 alone is itself a good diagnostic indicator. When the product of IL-27 and ADA is used to identify patients with TPE, a false-negative rate of 5% is seen.
The aetiological spectrum of pleural effusions is clearly dependent on the population studied, on the aetiological classification used and on the diagnostic methods employed. In developing countries the common causes of pleural effusions in adults are tuberculosis and bacterial infection,33 whereas in developed countries more than 90% of all pleural effusions are caused by congestive heart failure, malignancy, pneumonia and pulmonary embolism.34 More recently, we have reported that of 833 patients with undiagnosed exudative pleural effusion who underwent medical thoracoscopy, 333 (40.0%) were eventually diagnosed as having TPE.35 In contrast, the proportions of TPE are very low in countries with low levels of tuberculosis, such as New Zealand (5.9%),36 United Kingdom (5.6%),37 Spain (3.2%),38 France (2.7%),39 Denmark (2.1%),40 and United States (0%).41 We noted in our study that the diagnostic accuracy of IL-27 was better than that reported in the two European studies.8 9 The higher diagnostic accuracy of IL-27 seen in areas with a high incidence of tuberculosis is probably due to the increased prevalence of TPE.
Therefore, it is important to consider the pre-test probability of TPE in interpreting IL-27 results for clinical decision-making. Our meta-analysis indicated that when the predicted probability of TPE is low (10%), the overall post-test probability of positive results of IL-27 is high (76.7%), making it unacceptable for ‘ruling in’ TPE. However, in high prevalence settings a positive test result increases the probability for TPE up to 96.8%, meaning that further diagnostic procedures can be avoided. In all prevalence settings, when the result of IL-27 is negative, the post-test probabilities are sufficiently low (always <0.1%) to reliably rule out the disease, thus warranting the use of a biomarker-based approach.
Our meta-analysis had several limitations. First, a marked deficiency in the included studies is the use of narrow patient spectrums in the control group in the three studies conducted in China and a small proportion of patients with TPE included in the two studies conducted in Europe. These factors might have biased the estimation of test accuracy.42 43 Second, the number of studies available for inclusion in this meta-analysis was limited, with one study involving only 10 TPE specimens,9 and such small studies may be vulnerable to selection bias. Third, there might have been misclassification bias. A small proportion of patients with TPE were not diagnosed on the basis of bacteriological or histological results, but just on the clinical course.6 7 9 This might have caused non-random misclassification, leading to biased results. Fourth, control populations were heterogeneous among the studies; and various cut-off points were used for distinguishing between TPE and non-TPEs according to the best combination of sensitivity and specificity. These problems with the accuracy of diagnosis might also lead to biased results.
It should be mentioned that because the determination of ADA is simpler and less expensive, the current guideline recommends it as the preferred test for the diagnosis of TPE.32 In contrast, despite the superiority of IFN-γ in diagnosing TPE, the high cost and the lack of an accepted cut-off value for the discrimination of TPE limit its daily clinical use.2 27 Based on the excellent diagnostic performance of IL-27 for TPE found in this study, it is reasonable to recommend the application of pleural IL-27 measurement in routine clinical practice.
In conclusion, our prospective study findings have provided sufficient evidence to support the use of IL-27 as a valuable indicator for the diagnosis of TPE, with a diagnostic performance comparable to that of IFN-γ and significantly more accurate than that of ADA. Our meta-analysis data further support the use of IL-27 to rule in TPE in an area with high prevalence, and can also be reliably used as a ‘rule out’ test in all prevalence settings. It should be noted that since disparate cut-off values exist in different studies included in the meta-analysis, more extensive studies are needed to confirm these results, especially to establish an acceptable cut-off point for clinical practice.
We thank Mr Dian-Bo Liu for his excellent technical assistance with measurements of the soluble biomarkers. We thank physicians and nurses at the participating departments for inclusion of patients in this study. We also thank the patients for their participation in this study.
WW and QZ contributed equally.
Contributors H-ZS conceived the study and critically revised the manuscript. WW, QZ, KZ, YW and J-YL recruited and assessed the patients, and preprocessed the study specimens. X-JW and ZW were responsible for statistical analysis. J-CZ and Z-HT performed the meta-analysis. WW and QZ interpreted the data and wrote the manuscript. All authors contributed to the subsequent drafts and approved the final version.
Funding This work was supported in part by grants from National Natural Science Foundation of China (No. 31470883, No. 91442109, No. 81270149 and No. 81470274), in part by the Beijing Municipal Administration of Hospitals’ Mission Plan (No. SML20150301), and in part by Chinese Ministry of Science and Technology for the establishment of GCP evaluation system in respiratory diseases (2014ZX09303302).
Competing interests None declared.
Patient consent Obtained.
Ethics approval Institutional review board for human studies of Beijing Chaoyang Hospital; and institutional review board for human studies of Wuhan Union Hospital, China.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.