Interobserver-variability of lung nodule volumetry considering different segmentation algorithms and observer training levels
Introduction
Modern radiology has to cope with an increasing number of incidentally detected subcentimeter pulmonary nodules (SCPN) of unknown dignity. One reason for this development is the introduction of multislice computed tomography (MSCT) that led to an improvement of spatial resolution in daily routine CT examinations [1], [2], [3], [4]. Additionally, it can be expected that a growing number of undetermined SCPN is detected due to the progressive use of computer aided detection (CAD) software [5].
In a screening population the majority (95–98%) of SCPN is reported to be benign [1], [2], [6]. On the other hand, lung cancer screening studies with low-dose CT revealed a prevalence of asymptomatic cancers in 1.3% of a smoking population [2]. Therefore, it is mandatory to reveal lesion dignity for SCPN detected in an oncological setting or a risk-group screening. As malignancy is very rare in solid lesions of less than 5 mm in diameter [6] and lesions of more than 10 mm in diameter are usually surgically removed, a special interest is focused on lesion sizes from 5 to 10 mm. A general resection or biopsy of all detected SCPN in this subgroup cannot be recommended following ethical and economical considerations. Therefore, the interest focuses on non-invasive diagnostic methods [7].
The surveillance of nodule growth according to the RECIST criteria (assessment of maximum uni-dimensional lesion diameter) with physical or digital callipers in repetitive CT examinations is a widespread diagnostic tool [8]. Since growth must be present in all cancerous lesions and diameter based size measurements are easily performed by any radiologist, growth criteria are in more widespread use than other suggested features to determine lesion malignancy (e.g. shape, margin form, texture or enhancement). Unfortunately, especially small lesions can show significant volumetric changes while diameter changes may still not be detectable or of diagnostic significance [9]. Manual diameter measurements are also subject to non-neglectable interobserver variance [9], [10], [11]. Newly developed volumetric software tools proofed already a high accuracy [12], [13], reproducibility [14], [15] and satisfying interobserver variability [11], [14] for small solid lesions. A remaining potential source of uncertainty is observer experience. In everyday radiological routine it cannot be ensured that follow-up volume analyses are always performed by radiologists of equivalent experience. However, a consistent quality of volume quantification is of great importance in oncological settings to determine tumor response or in order to characterize lesions by growth-rate determination. Therefore, it would be of special interest to know if observer experience induced variances could be reduced by the choice of suitable software-tools.
The purpose of this study was to investigate, to which degree volumetric and diameter measurements are affected by observer experience, training effect or improved software algorithms.
Section snippets
The ex vivo system (“chest phantom”)
For this study, we used an existing ex vivo system for imaging porcine heart and lung explants. This system uses a polymer container constructed to simulate a chest that holds the freshly excised, inflated lung explant of a pig by continuous evacuation of the artificial pleural space with 20–30 hPa. The original model was modified to allow for repetitive canulations of the inflated lungs (Fig. 1) [16].
The heart–lung explants were harvested from pigs (80–100 kg) at a local slaughterhouse. No
Morphology
The two observer consensus evaluation (H.B. and J.B.) revealed that 65.7% (n = 46) of the artificial lesions represented morphological attributes of small, solid lung nodules with a roundish and slightly. Only this portion was used for further volumetric analysis. The median nodule diameter was 8.1 mm ranging from 5.4 to 11.1 mm, while 50% of lesions had a size between 7.2 mm (Q25) and 8.5 mm (Q75). The median nodule density was 71.2 Hounsfield units (HU) ranging from 8 to 137 HU with quartile values
Discussion
Previous studies could already prove a high accuracy and reproducibility and a low interobserver variability volumetrically determined lesion sizes [12], [13], [14], [16], [17], [18]. Remaining potential sources of possible uncertainties are observer experience and the chosen software system. In radiological routine follow-up volume analyses are frequently performed by radiologists of different experience levels. A consistent quality of volume quantification is of great importance in
Conclusions
Our study confirmed the finding of previous investigations that interobserver variance is significantly smaller for volumetric measurements than for unidimensional diameter measurements. In addition, interobserver variances of volumetric analyses are independent of observer experience. In an unexperienced observer group the interobserver variance of diameter measurements is significantly improved by a training effect. Advanced volumetric software tools that allow for “click-point robust”
References (21)
- et al.
Early lung cancer action project: overall design and findings from baseline screening
Lancet
(1999) - et al.
Lung cancer screening with low-dose CT
Eur J Radiol
(2003) - et al.
Early lung-cancer detection with spiral CT and positron emission tomography in heavy smokers: 2-year results
Lancet
(2003) - et al.
Managing the small pulmonary nodule discovered by CT
Chest
(2004) - et al.
Detection of pulmonary nodules by computed tomography: improved detection rate with reduced slice thickness
Eur Radiol
(2003) - et al.
Diagnostic performance of a commercially available computer-aided diagnosis system for semi-automatic detection of pulmonary nodules: comparison with single and double reading
Fortschr Röntgenstr
(2004) - et al.
Characterization of small pulmonary nodules by CT
Eur Radiol
(2004) - et al.
New Guideline to evaluate the response to treatment in solid tumors
J Natl Cancer Inst
(2000) - et al.
Are two-dimensional measurements of small noncalcified pulmonary nodules reliable?
Radiology
(2004) - et al.
Interobserver and intraobserver variability in measurement of non-small-cell carcinoma lung lesions: implications for assessment of tumor response
J Clin Oncol
(2003)
Cited by (38)
Multi-scale deep networks and regression forests for direct bi-ventricular volume estimation
2016, Medical Image AnalysisCitation Excerpt :The proposed estimation framework can be easily extended to other applications for volume estimation, such as estimating the volume of a tumor (Bolte et al., 2007; Heckel et al., 2014) which is still dependent on segmentation, and can also be used for extensive model parameter estimation problems, e.g., model personalization (Marchesseau et al., 2013; Zettinig et al., 2014).
Advanced imaging tools in pulmonary nodule detection and surveillance
2016, Clinical ImagingCitation Excerpt :However, it has been shown in a large screening trial and meta-analysis that nodules less than 5 mm tend to be benign, even in the high-risk population [34]. Moreover, Bolte et al. has shown that volumetry achieves significantly less interobserver variance compared to diameter measurements and advanced volumetry algorithms are independent of observer experience [35]. In fact, such a finding was confirmed by other recent studies [36].
Measurement of tumor volumes improves RECIST-based response assessments in advanced lung cancer
2012, Translational OncologyBreathe New Life Into Your Chest CT Exams: Using Advanced Acquisition and Postprocessing Techniques
2019, Current Problems in Diagnostic RadiologyCitation Excerpt :However, it has been shown in large clinical trials that nodules less than 5 mm tend to be benign, even in the high-risk population.40 Several studies have shown that volumetry achieves significantly less interobserver variance compared to diameter measurements and advanced volumetry algorithms are independent of observer experience,41,42 with reproducibility and accuracy preserved even with different dose settings and iterative reconstruction, which is often used for low dose or ultra-low-dose settings.43-48 According to Knoss et al, density measurements based on volumetry are more robust than regular 2-dimension measurements.