Chest
Volume 114, Issue 6, December 1998, Pages 1740-1748
Journal home page for Chest

Occupational and Environmental Lung Disease
Variability in the Classification of Radiographs Using the 1980 International Labor Organization Classification for Pneumoconioses

https://doi.org/10.1378/chest.114.6.1740Get rights and content

This study describes the extent of agreement in classification of chest radiographs using the International Labor Organization (ILO) classification among six readers from the United States and Canada. A set of 119 radiographs was created and read by three Canadian and three US readers. The two ratings of interest were profusion (scored from 0/-to 3/+) and pleural abnormalities consistent with pneumoconiosis (scored with the ILO system, then collapsed into a yes/no). We used a number of approaches to evaluate interreader agreement on profusion and pleural changes, determining concordance, observed agreement, kappa statistic, and a new measure to approximate sensitivity and specificity. This study found that five of six readers had good fair to good agreement for pleural findings and for profusion as a dichotomous variable (≥ 1/0 vs ≤ 0/1) using the kappa statistic, while a sixth reader had poor agreement. We found that concordance, expressed as percent agreement, was higher for normal radiographs than for ones that showed disease, and describe the use of the kappa statistic to control for this finding. This analysis adds to the existing literature with the use of the kappa statistic, and by presenting a new measure for “underreading” and “overreading” tendencies.

Section snippets

Background

Sheet metal work includes a variety of tasks entailing fabrication or installation of metal products. Respiratory hazards identified with sheet metal work include exposure to asbestos, welding fumes, and man-made vitreous fibers, primarily fiberglass. Three investigations have specifically studied asbestos-related disease among sheet metal workers employed in construction.18, 19, 20, 21 The most recent of these studies was sponsored by the Sheet Metal Occupational Health Institute Trust.

Results

Materials and Methods

A set of 119 radiographs was created and read by three Canadian and three US readers. The three Canadian readers and three US readers who had interpreted the largest number of radiographs for the program were asked to participate; initially, all six agreed to do so. One of the US readers was ultimately unable to fulfill this commitment, and an additional reader from the US program was asked to contribute at a later date. Each of the participating US readers had read > 350 radiographs for the

Concordance

Of the 119 radiographs in the study set, 114 had complete data from all readers for pleural changes, and 110 had complete data for profusion. Table 1 presents the distribution of pleural changes and profusion ratings for each of the six readers. The percent of radiographs positive for pleural changes ranged from 19 to 80%, with reader 1 having the highest proportion of abnormal radiographs and readers 4 and 5 reading the fewest pleural changes. For profusion, the proportion of radiographs with

Discussion

This study showed that five of six readers had good fair to good agreement for recording pleural findings and for profusion as a dichotomous variable (≥ 1/0 vs ≤ 0/1), while a sixth reader had poor agreement. We found that concordance, expressed as percent agreement with the group median, was higher for normal radiographs than for ones that showed disease, and illustrates the use the kappa statistic to test this finding. We also present each reader's sensitivity and specificity (each reader

References (31)

  • HjortsbergU et al.

    Railroad workers with pleural plaques

    Am J Ind Med

    (1988)
  • SprinceNL et al.

    Asbestos-related disease in plumbers and pipefitters employed in building construction

    J Occup Med

    (1985)
  • SchwartzDA et al.

    Asbestos-induced pleural fibrosis and impaired lung function

    Am Rev Respir Dis

    (1990)
  • NicholsonWJ et al.

    Occupational exposure to asbestos: population at risk and projected mortality-1980-2003

    Am J Ind Med

    (1982)
  • LilienfeldDE et al.

    Projection of asbestos related diseases in the United States, 1985–2009 I

    Cancer. Br J Ind Med

    (1988)
  • BecklakeMR

    Asbestos-related diseases of the lungs and pleura: current clinical issues

    Am Rev Respir Dis

    (1982)
  • Guidelines for the use of ILO international classification of radiographs for pneumoconioses

    Occup Safety Health Series

    (1980)
  • MorganRH

    Proficiency examination for classification of pneumoconiosis chest films

    AJR Am J Roentgenol

    (1979)
  • MuirDCF et al.

    Classification of radiographs for pneumoconiosis: the Canadian pneumoconiosis reading panel

    Am J Ind Med

    (1993)
  • FletcherCM et al.

    The problem of consistent radiological diagnosis in coal miners' pneumoconiosis

    Br J Ind Med

    (1949)
  • AshfordJR

    The classification of chest radiographs for coal workers pneumoconiosis: a study of the performance of two readers over a period of 6 years

    Br J Ind Med

    (1960)
  • FelsonB et al.

    Observations on the results of multiple readings of chest films in coal miners' pneumoconiosis

    Radiology

    (1973)
  • AttfieldMD et al.

    An investigation of inter-reader variability among xray readers employed in the underground coal miner surveillance program

    Ann Amer Conf Gov Ind Hyg

    (1986)
  • RossiterCE et al.

    International classification trial of AIA set of 100 radiographs of asbestos workers

    Br J Ind Med

    (1988)
  • JacobsenM et al.

    A study of a sample of chest radiographs from the Health and Safety Executive's survey of asbestos workers

    Report No. TM/83/4, Institute of Occupational Medicine, Edinburgh, Great Britain

    (1983)
  • Cited by (0)

    Funded in part by the Sheet Metal Occupational Health Institute Trust, a trust jointly funded by the Sheet Metal Workers International Union and the Sheet Metal and Air Conditioning National Association.

    View full text