Background: A World Health Organization (WHO) working group in 2001 developed a method for standardizing interpretation of chest radiographs in children for epidemiologic purposes. We reevaluated radiographs from the Kaiser Permanente Pneumococcal Efficacy trial using this method.
Methods: Seven-valent pneumococcal conjugate vaccine was evaluated in a randomized, controlled study including 37,868 infants. Effectiveness against pneumonia was previously evaluated using the original treating radiologist reading. There were 2841 sets of radiographs from this trial and all available radiographs were scanned and read blindly by 2 WHO crosstrained readers (A and B); discordance between the 2 primary readers was resolved through a consensus reading by an adjudicating panel of 2 radiologists.
Results: Of the 2841 radiographs, 2446 were available for scanning and were reviewed using WHO-defined descriptive categories. Two hundred fifty of the 2446 radiographs were read as positive by both readers. An additional 129 were read as positive by reader A only and 142 by reader B only for a total of 521 radiographs that were read as positive by one or both of the reviewers. The concordance rate between the 2 reviewers was 250 of 521 (48%). Of the 271 discordant radiographs, 45 of 129 (34.9%) of reader A and 66 of 142 (46.5%) for reader B were finalized as positive by the adjudicating panel. Overall, 361 radiographs were finalized as positive (12.7%). With these 361 images as the standard, the sensitivity and specificity of reader A were 82% and 97%, respectively, and for reader B, 88% and 97%, respectively. Kappa between the 2 readers was 0.58. Of 25 control radiographs read as positive by both A and B, 80% were also read as positive by the panel and all 25 control negative radiographs were read as negative by the panel. Using original readings by point-of-care radiologists, efficacy against first episode of radiograph confirmed pneumonia was 17.7% (95% confidence interval [CI] = 4.8-28.9%) in intent-to-treat and 20.5% (95% CI = 4.4-34%) in per protocol. Using the WHO method, the efficacy against first episode of radiograph confirmed pneumonia adjusting for age, gender and year of vaccination of 25.5% (95% CI = 6.5-40.7%, P = 0.011) for intent-to-treat and 30.3% (95% CI = 10.7-45.7%, P = 0.0043) for per protocol.
Conclusion: Using WHO criteria for reading of radiographs increased point estimates of vaccine efficacy presumably as a result of improved specificity.