Machine learning methods on exhaled volatile organic compounds for distinguishing COPD patients from healthy controls

Subjects gave written consent and the study was approved by the local ethics committee and registered (ISRCTN 82911859). COPD patients were identified through hospital and primary care registers. All were deemed stable by a respiratory clinician and none reported worsening symptoms within six weeks of testing. All were prescribed optimal medication [2]. Healthy controls comprised spouses of patients, volunteers from local charity organizations and members of staff with no chest pain, breathlessness, cough or wheeze on screening questions.

Anyone who did not respond to a written and telephone invitation, refused consent, with current or previous cancer (except non-melanoma skin cancer), known dementia, heart failure, other known pulmonary, renal or liver disease or pregnancy was excluded. Participants completed questionnaires for socio-demographic data, smoking status and any illnesses including current/recent symptoms then performed dry wedge spirometry (Vitalograph Alpha®,Buckinghamshire, UK). Smoking status was validated using exhaled carbon monoxide (CO) (Bedfont-Micro Smokerlyzer®). An approved research standard [22] was used to define never, ex- and current smokers. Resting peripheral oxygen saturations on air (Konica Minolta Pulsox-300, Konica Minolta Sensing Inc, Osaka, Japan) and BMI were recorded.

The study comprised 119 patients with COPD and 63 healthy controls. COPD was observed in differing severities which are described according to internationally agreed guidelines from the global initiative for chronic obstructive lung disease. This is based on the severity of lung impairment (obstruction FEV1/FVC <0.70) where 'Mild COPD' is defined as GOLD Stage 1 and FEV1 = > 80% predicted, 'Moderate COPD' is defined as GOLD Stage 2 with FEV1 =>50% but <80% predicted, 'Severe COPD' is defined as GOLD Stage 3 with FEV1 = > 30% but <50% predicted, 'Very Severe COPD' is defined as GOLD Stage 4 with FEV1 =<30% predicted. Of the COPD group there were 7 GOLD stage 1, 48 GOLD stage 2, 38 GOLD stage 3 and 22 GOLD stage 4. In addition, four patients had an FEV1/FVC >0.70 but had symptoms of COPD, were on treatments and had emphysema on thoracic HRCT scans. Table 1 describes both groups.

The patient group is typical in age, smoking and demographics of COPD patients attending hospital. The control group is matched, as far as possible, for gender and age and is drawn from the same community (socioeconomic class). COPD is a smoking related illness and there was a clear imbalance of smoking status in between COPD and control groups as finding heavy smokers, of an appropriate age and who did not suffer from COPD or other lung dysfunction is difficult.

Given the imbalance of smoking status between the COPD and control groups, sub-analysis was also performed. The first sub-analysis involved dividing the COPD group into ex smokers and current smokers, so that the indicators of smoking could be compared with apparent indicators of COPD. The second sub-analysis eliminated current smokers from both groups in order to look for indicators of disease without the influence of smoking related VOCs. It was not possible to compare current smoking controls with current smoking COPD patients as there were insufficient numbers of the former to perform reliable analysis.

The same clinical room (with closed door and no air conditioning) within the hospital was used for sample collection. However, due to the practicalities of gathering the subjects together, sampling had to be performed on multiple days. After fasting for 4 h, subjects performed spirometry then rested for 20 min within the test room prior to breath collection. Patients were asked to stop short-acting bronchodilators and inhaled corticosteroids for at least 12 h, and any long-acting bronchodilator for 24 h before testing.

Breath samples were collected using a commercially available sampler (Bio-VOC®, Markes International Limited, Llantrisant, UK). This apparatus has been reported in a previous study [10]. Participants took a deep breath and then slowly exhaled through a sampler to their vital capacity until further exhalation was not possible. The sampler consists of a PTFE syringe which holds 129 mL of air with the initial dead-space air displaced by the alveolar air as exhalation proceeds; although there is no data on the breath distribution in these samplers. After full exhalation, a plunger is used to force the breath into a sorbent tube where VOCs are then trapped. Flow rate was not measured but is limited by the resistance offered by the sampler. Three individual samples were taken (i.e. three tubes per patient) with 2 min intervals between each. This allowed more data to be gathered for each subject than the single samples used in most studies. On each of the sampling days, a single air sample was taken in the room prior to testing for monitoring purposes but this was not used in the automatic data analysis. The sorbent tubes (stainless steel (Markes International Limited)) were preconditioned and contained two graphitized carbon black sorbents (Carbograph 1TD and Carbopack X, total sorbent mass of ∼470 mg). An airtight brass cap with a PTFE seal was fitted to the tube ends before storage in lined steel cans for transportation to the laboratory for processing within five days, in batches of typically 20 or more tubes.

Batches of tubes were loaded into an Ultra unit (Markes International, Llantrisant, UK) for automated processing via a Unity thermal desorption unit and fed with inert helium at 10 psi to desorb the tubes in a 30 mL min⁻¹ stream of inert helium at 300 °C for 5 min. To further concentrate, the flow was driven onto a cold trap (U-T11GPC, general purpose graphitized carbon C4/5-C30/32) set at −10 °C. The trap was then desorbed at 300 °C for 3 min. To allow sufficient flow through the trap, 5 mL of the sample is vented and the remainder injected into the GC (Agilent Technologies 6890N gas chromatograph). The VOCs were separated using a capillary column (30 m × 0.25 mm id, HP-5MS film thickness 0.25 µm). The column temperature was initially set at 40 °C and then increased steadily to 200 °C at a rate of 5 °C min⁻¹. Mass spectrometry (Agilent Technologies 5973 network mass selective detector) was used in electronic ionization mode and mass spectral data obtained in the SCAN mode with a mass range m/z 40 –550. Each tube was run twice to prevent carry over of volatiles onto the cold trap in subsequent samples and to check for VOCs inherent in the system.

Automated peak detection and baseline correction was used to calculate peak area (area under the curve) and retention time (RT) for each compound. Using an automated library search function in the Chemstation GC/MS software (Agilent Technologies), VOCs were tentatively identified using the NIST 98 mass spectral library (The National Institute of Standards and Technology) at the apex of each peak and utilizing probability-based matching. RTs were consistent for the vast majority of compounds over the course of the study; for example the standard deviations of RTs in toluene and undecane were 0.03 and 0.04 min, respectively.

Two methods were employed to normalize/scale the data. The first approach used total VOC levels to normalize each sample (i.e. the sum total magnitude of all VOCs in a particular sample—thus expressing each VOC as the proportion of total VOC). Total VOC was calculated for a RT window (1.49–24.5 min) to exclude noisy spectra at low RT and column bleed at high RT. In addition, known system peaks such as siloxanes were also omitted. This approach has been used previously by Van Berkel et al [16, 43] to compensate both for mixing of alveolar and upper airway breath and drifts in sensitivity of the test equipment. However, this study found that total VOC levels were substantially higher in COPD than control subjects (both in terms of the number and abundance of VOCs) so this approach effectively reduces the levels of individual VOCs in COPD, which may influence the ability to learn from the data. Therefore, an additional scaling method was used in which VOC levels were divided by the average area under the curve for benzaldehyde in the entire batch of measurements from which it originated (rather than individual samples). Normalization allowed for compensation to be made for drifts in cold trap and MS performance and removed the need for introducing internal standards into the sample. Benzaldehyde was present in almost all samples, and although it did vary from sample-to-sample, averages for both COPD and controls within a given batch were similar across the series of collections (see figure 1). Benzaldehyde has also been reported in other studies as being one of the most prevalent VOCs in breath [23] with van den Velde et al [24] suggesting that it is due to the environment. In this study, benzaldehyde was detected in over 90% of air samples, and at higher levels than in breath, suggesting an environmental source. Analysis of sorbent tubes injected with controlled amounts of toluene, xylene and dodecane demonstrated that the system gave very similar mass responses for each compound. The RTs of these compounds covered the range of most of the detected compounds.

Zoom In Zoom Out Reset image size

In the field of machine learning, there are a large range of different techniques for supervised learning [25], each with its own underlying mathematical foundation or metric (for example fuzzy sets [26], support vector machines [27], Bayes, entropy, etc), and each with its own merits and demerits. Some techniques may perform better than others, and depending on the data, one approach may outperform another, however the same may not be true given a different dataset [25]. As such, in order to provide a comprehensive and robust evaluation, a number of approaches have been included for the data analysis. In addition to techniques which have different underlying theories, a number of different approaches to learning such as ensembles [25] (where multiple classifiers are combined) are also included. As well as different learning techniques, an additional step known as feature subset selection is also incorporated. The main aim of FS is to determine a minimal feature subset from a problem domain whilst retaining a suitably high accuracy in representing the original features. The usefulness of a feature subset is determined by both its relevance and redundancy. A feature is said to be relevant if it is predictive of the decision feature(s), otherwise it is irrelevant. A feature is considered to be redundant if it is highly correlated with other features. Hence, the search for a good feature subset involves finding those features that are highly correlated with the decision feature(s), but are not correlated with each other. FS attempts to focus selectively on relevant features, whilst simultaneously attempting to ignore the (possibly misleading) contribution of irrelevant features. It should be noted that the approaches employed in this paper select subsets of features rather than individual features e.g. based on the rank of the individual features.

All detected VOCs from 546 data objects (119 × 3 = 357 for COPD patients, and 63 × 3 = 189 for controls) were examined. The rationale for this investigation was to determine what results could be obtained without any subjective human input such as a tunable threshold, or prior domain knowledge which may affect or optimistically bias the learning process. All of these data objects (individual patients/controls) retain their link to the original patient/control for the purpose of classifier learning such that data objects which are related to the same subject do not appear simultaneously in both training and testing phases.

For analysis of the data, there were two steps involved: firstly, classifier learning based on the full set of VOCs (shown in figure 2) and secondly, classifier learning based on an automatically selected subset of the VOCs using FS (shown in figure 3). It is important to note that the feature subsets obtained using FS were obtained independently of any classifier learning technique, i.e. the learning techniques were not used as a metric for the selection of subsets of features (this can often lead to overfitting). Experiments were carried out using in-house software in partial combination with Weka [28] and some stand-alone software. Weka is an open-source collection of machine learning algorithms for data mining tasks and contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. It can also be used as a platform to develop new machine learning schemes.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Overall, 12 different automatic classifier methods were used. These were based on various different techniques and schemes; with eight stand-alone classifiers, and two ensemble techniques (both used with different base classifiers). As mentioned previously, the reasons for including a variety of learning methods is to provide a comprehensive and robust examination of the data. The learning methods comprise: J48 (a version of Quinlan's ID3 decision-tree algorithm) [29], JRIP (a rule-based 'ripper' classifier which generates rules and uses these to classify 'new' objects) [30], PART (another rule-based classifier) [31], SMO (sequential minimal optimization approach for support vector machines) [27], FNN (fuzzy nearest-neighbour classifier) [32], random forest (a tree-based classifier) [33], FRNN (a fuzzy-rough set nearest-neighbour approach) [34] and VQNN (a noise-tolerant fuzzy-rough set-based classifier) [35]. For the ensemble techniques, J48 and VQNN were used as base classifiers, while random subspace [36] and bagging [37] schemes were implemented with a redundancy elimination step.

For testing and validation of the classifier learners, stratified ten times 10-fold cross-validation (10-FCV) is employed. The original dataset is partitioned into ten subsets and of these ten subsets, a single subset is retained as the testing data for the classifier, and the remaining nine subsets are used for training. The cross-validation process is then repeated ten times (the number of folds). This process is then repeated ten times, resulting in 100 runs in total. Note that there are other available training/validation and testing schemes which are equally suited to this task, however they are not implemented for the study in this paper. The stratification of the data prior to its division into folds ensures that each class label (as far as possible) has equal representation in all folds, The process of stratification (in addition to the use of 10-FCV) helps to further alleviate the problems of class-imbalanced data, and thus bias/variance [38]. The 10-FCV validation strategy produces very robust results and good generalizability [38–40] when compared with strategies such as leave-one-out cross-validation, which suffers from the problem of large variance [38]. The large variance is a result of predictions which are generated using almost all of the data (only one object is 'left-out' for each cross-validation), i.e. a small change in data causes a large change in the generated model (over-fitting) [38, 39].

For the FS step, an approach based on fuzzy-rough sets is employed [41]. This approach is a filter-based method, which is entirely data-driven and does not require any subjective thresholding values or have any tunable parameters, nor does it require any prior domain knowledge. A number of search strategies were adopted for the discovery of subsets (i.e. informative VOCs): (1) Greedy hill-climbing—where features are added greedily until a stopping criterion is reached. (2) SAT (propositional satisfiability)—subsets are generated using a SAT solver or constraint satisfaction problem [42]. (3) This approach is a variant of approach 1 above, where a tenfold cross-model-validation is employed for the generation of feature subsets. The intersection of all subsets could then used as the final reduct or subset. It should be noted that features are selected at each round of cross-validation described previously, thus providing a cross-model-validation of the data.

The performance of the various techniques was measured using classification accuracy, area under the receiver operating characteristic curve (AUC) as well as sensitivity (the proportion of correct predictions for COPD subjects) and specificity (the proportion of correct predictions for control subjects).

The approach was repeated for all of the sub-analyses leading to three comparisons:

• (1)  
  COPD versus control for all subjects.
• (2)  
  Current versus ex smokers for COPD subjects.
• (3)  
  COPD versus control with current smokers removed.

In general, higher accuracies were observed when the data was normalized with respect to total VOC than when normalized by using benzaldehyde. Prior to the application of FS, and using total VOC normalization, the most accurate individual classifier for COPD versus control was PART (a rule-based classifier) with an overall classification accuracy of 71.6% (81% of COPD group correctly predicted and 54% correct prediction for control subjects) and an AUC of 0.71. When data was normalized with respect to benzaldehyde, the maximum accuracy and AUC were 71.5% and 0.78, respectively (for Random SubSpace/J48 ensemble). The application of FS tended to result in improved performance when normalizing using total VOC, and overall accuracy increased to 73.8% (79% for COPD and 64% for control) with VQNN as the best individual classifier learner resulting in an AUC of 0.79. A maximum AUC value of 0.82 was achieved with an ensemble employing VQNN with bagging and a redundancy removal step for the reduced data; this gave an accuracy of 73.4% (82% for COPD and 58% for control) at this point. When data was normalized using benzaldehyde, the use of FS tended to result in slightly reduced accuracy for the better performing methods, but increased performance in the poorly performing methods. Maximum accuracy and AUC were 70.5% and 0.76, respectively, using a Bagging/J48 ensemble. Summary data for all the methods, including standard deviations, is provided in the online supplementary material, available from stacks.iop.org/JBR/6/036003/mmedia.

The subset of selected VOCs, automatically generated from the complete dataset following FS is listed in table 2 for data normalized by both total VOC and benzaldehyde. The dataset for which they were selected (total VOC normalized, benzaldehyde normalized, or both), the frequency at which they were detected in COPD and control subjects and their mean abundance over all samples (normalized with respect to both total VOC and batch data for benzaldehyde) together with the standard error are shown. Match quality and RTs are also shown. Most selected VOCs identified had high quality scores. However, butane and carbon dioxide suffered from poor match quality resulting from their low molecular weight but they did have consistent mass fragments and RTs, suggesting that the same compound was being identified each time. In addition spiking a thermal desorption tube with pure butane gave similar mass fragments and RT to the breath data. This approach did not tend to identify compounds that appeared in only a small numbers of subjects; with most compounds present in the majority of both COPD and control groups. The exception was phthalic anhydride which occurred in less than 18% of COPD subjects and only in a single sample in one of the controls. Furthermore, the compounds benzene, hexanal, nonadecane, and toluene all appear in the subsets for each of the three breath samples, when FS is performed on this data in isolation. Different feature subsets were selected, depending on the method of normalization, although both methods gave a core (that were found in all ten cross-model-validations) of common VOCs (Isoprene, Acetic acid, benzaldehyde, benzene, butane, nonadecane and toluene). Normalization of the data using benzaldehyde resulted in smaller feature subsets being selected and did not, for example, result in subsets containing hexanal or phenol.

Classification accuracy was higher when comparing current versus ex smokers in the COPD group than when attempting to classify COPD and control groups. For the unreduced data (without FS), the best accuracy achieved was 87.6% (81% of current smokers correctly predicted and 91% of ex smokers) when using a Random SubSpace/J48 ensemble on the benzaldehyde normalized data. An optimum AUC of 0.94 was achieved using Bagging with J48. These were reduced slightly for total VOC normalized data; where the optimum accuracy and AUC were: 85.3% (80% of current smokers correctly predicted and 91% of ex smokers) and 0.92 when using the random forest classifier. When FS was employed on the total VOC normalized data, there was a slight decrease in overall classification accuracy following FS to 83.7% (82% of current smokers correctly predicted and 87% of ex smokers) with an AUC of 0.90 (both achieved using random forest). There was a more substantial decrease in performance when FS was employed on the benzaldehyde normalized data; with classification accuracy and AUC of 79.9% and 0.83, respectively (achieved using JRIP and Bagging with J48, respectively). Summary data for all the methods, including standard deviations, is provided in the online supplementary material, available from stacks.iop.org/JBR/6/036003/mmedia.

The subset obtained using FS contained the compounds listed in table 3. The subsets of compounds have substantial overlap with those discovered for the COPD versus control experiments (Isoprene, Benzaldehyde, Benzene, Toluene, Acetic Acid, Carbon dioxide, Sulphur dioxide). Only 1-Heptene, and 1,3,5-Cycloheptatriene were not in the COPD versus control FS and the latter has a similar RT to toluene which, together with the poor quality score, suggests that it may have been misidentified. Again, different features were selected, depending on the normalization method employed, though both methods gave a core of common VOCs. The similarity of compounds derived using FS in smoking with those in COPD versus control suggest that the original task of differentiating COPD from healthy controls is influenced by the smoking status of subjects.

For the comparison of COPD and control subjects with current smokers omitted, and normalized using total VOC, classification accuracy was improved for all of the models tested when compared to the overall study of all subjects. The best performing technique was an ensemble of Bagging with J48 as the base classifier. This gave a classification accuracy of 74.3% (79% correct classification for COPD and 68% for controls) and an AUC 0.82. When normalizing using benzaldehyde, optimum accuracy and AUC were obtained using the same classifier but reduced to 67.1% and 0.73, respectively. With FS, the same ensemble again gave optimum results and there was an increase in classification accuracy to 76.0% (78% correct classification for COPD and 74% for controls) and an AUC of 0.84. When normalizing using benzaldehyde, optimum accuracy and AUC were obtained using the same classifier but reduced to 68.1% and 0.72, respectively. The results for total VOC normalized data show a slight improvement over those obtained for COPD versus control with all subjects included, particularly in terms of specificity (prediction of controls). However, the amount of data available for learning is smaller which will potentially reduce accuracy. Also, the control group contains subjects who have never smoked; smoking history, like other lifestyle factors is presumed to affect VOCs. Summary data for all the methods, including standard deviations, is once again provided in the online supplementary material, available from stacks.iop.org/JBR/6/036003/mmedia.

The subsets obtained using FS contained the compounds listed in table 4. Isoprene, acetic acid, benzaldehyde, benzene, butane and carbon dioxide were selected for both normalized datasets. However, toluene, which appeared in the selected subsets when analysing smoking and when comparing all COPD and patients, was not selected by the FS method when smokers were omitted. Otherwise, there was much similarity with the compounds in the other parts of the study, indicating that there is a similarity between indicators of smoking status and indicators of disease. Cyclopentanone and ethaethioamide were not accurately identified but had consistent RTs. They also occurred in very few samples and should be regarded with suspicion.