Scoring of online cases: Interobserver agreement for the diagnosis categories, ‘UIP’, ‘possible UIP’ and ‘inconsistent with UIP’ expressed as Cohen's weighted κ coefficient stratified according to observer experience and specialty
| Interobserver agreement | ||
|---|---|---|
| Mean±SD | Median (IQR) | |
| UIP diagnosis categories (UIP, possible UIP, inconsistent with UIP) | ||
| Thoracic radiology fellows (n=5) | 0.47±0.05 | 0.50 (0.10) |
| Thoracic radiologists (experience <10 years, n=42) | 0.50±0.12 | 0.51 (0.16) |
| Thoracic radiologists (experience 10–20 years, n=27) | 0.51±0.11 | 0.52 (0.20) |
| Thoracic radiologists (experience >20 years, n=22) | 0.48±0.14 | 0.51 (0.18) |
| General radiologists (n=16) | 0.45±0.13 | 0.48 (0.18) |
| Binary diagnosis score (Typical UIP or Possible UIP/inconsistent with UIP) | ||
| Thoracic radiology fellows (n=5) | 0.36* | |
| Thoracic radiologists (experience <10 years, n=42) | 0.42* | |
| Thoracic radiologists (experience 10–20 years, n=27) | 0.39* | |
| Thoracic radiologists (experience >20 years, n=22) | 0.40* | |
| General radiologists (n=16) | 0.41* | |
The ‘possible UIP’ and ‘inconsistent with UIP’ categories were combined to generate a binary ‘typical UIP or possible UIP/inconsistent with UIP’ score. Interobserver agreement expressed as Cohen's κ coefficient for this binary categorisation, stratified according to observer experience and specialty.
*Unweighted κ.
UIP, usual interstitial pneumonia.