METHODS

Children aged 2–10 years were recruited from local schools (controls) and from ambulatory and outpatient clinics (coughers and wheezers). Within-occasion repeatability described in absolute values of Rint (kPa/l.s) has been reported previously in children aged 2–5 years,⁵ but measurements related to normative data are reanalysed here together with those of older children to examine the effect of age over a wide range and the effect of health status on repeatability.

The children comprised three groups: (1) those with no history of respiratory symptoms by accepted criteria⁶; (2) those with persistent isolated cough (who had had cough for more than 3 weeks or on three occasions in the previous 6 months); and (3) those with doctor observed wheeze in the previous 4–6 weeks but who were not wheezy at the time of testing. No child was on long term treatment. Children with upper respiratory tract infections in the previous 3 weeks were excluded.

Rint was measured as previously described⁵ in the expiratory phase of tidal breathing using a Micromedical device. If the coefficient of variation of the values contributing to each measurement was more than 20%, the measurement was excluded. During the same laboratory visit three measurements were made 15 minutes apart, before and after placebo and salbutamol 400 μg given in random order to which the observer was blind. Both were given using a spacer device and inhaled during tidal breathing. Only measurements of those who received placebo first were analysed.

For between-occasion repeatability, those children who had successfully undertaken measurement of Rint but not necessarily the 15 minute repeatability testing and were willing to return had a further test within the following 20 weeks. We are not able to describe those children whose parents did not give permission for them to be studied as we could not obtain the necessary information, but we have no reason to believe that subjects whose parents agreed to their participation were any different from those whose parents did not. The coughers returned because they were still coughing. Measurements were made at a similar time of day either in the routine lung function laboratory (essentially an ambulatory setting⁵) or in local schools. Both the researcher and subject were blinded to the results of the first measurement.

The local ethics committee approved the project and parents and children old enough to understand the project gave informed consent for the study.

Data analysis

Data were analysed using Nanostat (AlphaBridge Ltd, London, UK). Using normative data from the local population,⁷ measurements were expressed as the percentage expected for age because between-occasion measurements could be up to 20 weeks apart. For our data, age predicts Rint slightly better than height. The equation used was:

Repeatability was described as follows:

1. The limits of agreement of two measurements = 2 standard deviations (SD) of the differences between measurements. This was calculated for measurements (percentage expected) before and after placebo for the within-occasion estimation and similarly for paired baseline between-occasion measurements. So that the results could be compared with some of those published, the limits of agreement of unadjusted values (kPa/1.s) are also quoted. Change expressed as a multiple of the SD of the mean value for age (z score) is also presented. If a difference between two measurements lies between the limits of agreement, it cannot be said with 95% certainty that there has been “true” change. If a difference lies above these limits, such a difference cannot be explained by measurement error alone.

2. The coefficient of variation of a measurement = (SD of the differences between measurements/√2)/mean of the measurements. The coefficient of variation of a measurement describes the precision of the measurement related to its mean value.

3. The intraclass correlation coefficient = 1 – ((SD differences between measurements/√2)²/(SD of the measurements²)). The intraclass correlation coefficient describes how well pairs of measurements correlate.

The relationship of the limits of agreement with health status was examined by calculating the F ratios for the variances of the three groups (controls, coughers, and wheezers). This statistical test examines whether there is a significant difference between the variances of the groups. To examine the effect of age on the variance, the absolute residuals of all data were regressed on age and group.

RESULTS

Fifty five percent of the study participants were boys; 52% were Bangladeshi, 15% Afro-Caribbean, and 33% white British. Z scores for height were as previously published.⁷

Within-occasion repeatability

Of 174 children who agreed to undertake reversibility testing, 85 (46 aged 2–5 years and 39 aged >5–10 years) received the placebo first. There were 18 healthy children, 28 coughers, and 39 wheezers. The measurements of repeatability are shown in table 1. There was no relationship between the width of the limits of agreement and age (p=0.68) and health status (healthy v coughers, F ratio=1.22, p=0.31; healthy v wheezers, F ratio=1.1, p=0.38). The limits of agreement of the two measurements were 20% of expected Rint for age.

Following administration of bronchodilator, the median fall in Rint was 8.5% expected (interquartile range 3.5–12.7%) in the healthy children, 17.1% expected (interquartile range 5.9–28.3%) in coughers, and 22.6% expected (interquartile range 6.3–44.9%) in wheezers. Bronchodilator responsiveness in healthy children differed from that in both coughers (p=0.03) and wheezers (p=0.01). One of the 18 healthy children, 12 of the 28 coughers, and 22 of the 39 wheezers responded to salbutamol with a change of over 20% expected in Rint for age, the within-occasion limits of agreement.

Between-occasion repeatability

In 224 children who agreed to Rint measurements but not necessarily to the within-occasion study, the median time between measurements was 3 weeks (range 2–20). Their age ranges, health status, and measurements of repeatability are shown in table 1.

The F ratio for healthy children v coughers was 2.13 (p=0.001), for healthy children v wheezers was 2.64 (p<0.0001), and for coughers v wheezers was 1.23 (p=0.18). Multiregression analysis showed that, when health status was taken into account, the increase in standard deviation/year of age between ages 2 and 10 was no more than 2.3% (p=0.02).

Differences between tests before and after placebo were unrelated to their means (r²=0.003), which suggests that there was no “regression to the mean” for within-occasion measurements. There was no relationship between measurement differences and the time between tests (r²=0.05), suggesting that tests repeated within a short time were no more in agreement than those within a longer time.

DISCUSSION

This study has shown that within-occasion repeatability, as described by the limits of agreement, is 20% of expected Rint for age and is similar for healthy children, coughers and wheezers. This means that we cannot be 95% confident that any difference of less than 20% expected between two measurements made on the same occasion is true change. This figure must be interpreted together with changes that are likely to be encountered. In this study Rint changed by more than 20% expected in response to a bronchodilator in only one of 18 healthy children. As might have been predicted, in previous wheezers the response exceeded 20% expected in significantly more children. (For differences less than this, where differences in healthy and wheezy children in response to an intervention overlap, the interpretation of the difference is assisted by a receiver-operator characteristic curve.³ Discussion of this is outside the scope of this paper.) Coughers were in an intermediate position. Those with persistent wheezing or who were wheezy at the time of testing might be expected to have an even greater response.

For between-occasion repeatability, the limits of agreement for healthy children were 32% expected but, for stable children who had been wheezing within the previous 6 weeks, this rose to 52% expected. As a hallmark of asthma is bronchial lability, this is not surprising. However, this observation could be of diagnostic interest. If a child’s repeat measurement is outside the limits of agreement of differences in healthy children, then that child could be classified with those who are either coughers or wheezers. Children with isolated cough take an intermediate position. These results would fit in with our previous observations that bronchial lability measured by bronchodilator responsiveness in coughers is intermediate between that in controls and in wheezers.^3,4 Unexpectedly, there was a very small increase in standard deviation of the differences between measurements with age. Repeatability would be expected to be poorer in young children. This small increase is unlikely to be of clinical importance. There is also a small increase in the mean difference between occasions, but only about 10% of limits of agreement. We are unable to explain this as the equipment was the same between occasions and operators had undertaken acceptable interrater reliability measurements before embarking on the project.⁸

We have also presented intraclass correlation coefficients (ICC) for within-occasion and between-occasion repeatability, as these are sometimes quoted in the results of other studies. In this study the ICC for within-occasion repeatability of Rint was 0.97, suggesting nearly perfect correlation. The limits of agreement were 20% expected. For measurements of repeatability the limits of agreement are much more informative.

Clinicians often request lung function testing to monitor disease progress or the response to treatment in an individual. However, with Rint it seems that change between two occasions cannot be measured with confidence. In a controlled trial of the effect of corticosteroids on Rint in preschool children, those with positive skin prick test results were shown to benefit significantly after 6 weeks of treatment. Mean Rint improved by 16% for the group.⁹ However, the between-occasion repeatability of the measurement described in the present study suggests that, in the individual, this magnitude of change could not be detected with 95% confidence.

By comparison, the within-occasion coefficient of variation of forced expiratory volume in 1 second (FEV₁) in a group of healthy and asthmatic children is 4.3%,¹⁰ which means that the limits of agreement for change will be about 12%. (These figures are based on absolute values and not percentage expected.) Bronchodilator responsiveness measured using spirometric tests is considered positive if there is a change in FEV₁ of 12%.¹¹ However, for between-occasion repeatability the coefficient of variation of FEV₁ is 8.3% and the derived limits of agreement rise to 23%.¹⁰ In a clinical trial of the effect of inhaled corticosteroids, FEV₁ changed by a mean of 5% over 6 weeks.¹² This change would not be detected in the individual with confidence using spirometric tests.

For precise assessment of repeatability of any measurement (based on the 95% confidence limits of the estimate of variance), a minimum of 50 pairs of measurements is needed.¹³ Estimates of repeatability of measurements of airway resistance have been made by others using the interrupter technique with fewer measurements and using devices with different specifications.^14–17 Nevertheless, our measurements of coefficient of variation and intraclass correlation coefficients are comparable with those published. Ducharme and Davis¹⁸ measured airway resistance using the forced oscillation technique in 114 asthmatic children aged 3–17 years and obtained a within-occasion coefficient of variation of 9%. There are no between-occasion studies of adequate numbers of the repeatability of Rint, resistance measured using the forced oscillation technique, or resistance measured by plethysmography (Raw).

Van Noord et al¹⁹ measured Raw and FEV₁ on the same occasion in a group of asthmatic adults. Three measurements of Raw were followed by three measurements of FEV₁, from which were calculated the coefficients of variation of each and the limits of agreement derived (coefficient of variation × 2√2). The limits of agreement for FEV₁ were 13% and for Raw were 21%. Using these “thresholds”, a response to bronchodilator was demonstrated in more subjects using resistance than using FEV₁. Thus, although the resistance measurement seems to be more poorly repeatable, it is better for measuring change, at least in response to bronchodilators. A comparison of Rint, Raw, and FEV₁ in children has also shown that the coefficient of variation for FEV₁ is smaller than either Rint or Raw. However, measurements following challenge testing with methacholine found a larger change with Raw than the others when change was expressed as multiples of baseline standard deviation.¹⁵ The changes in Rint and FEV₁ expressed in the same way were similar. Thus, although the repeatability of FEV₁ appears better than that of resistance measurements, the change in response to challenge is detected as well or better by resistance measurements. How changes in resistance measurements compare with a change in FEV₁ in children in response to a bronchodilator is not known.

In summary, the limits of agreement for the measurement of Rint seem wide. When we consider the change expected following an intervention where an immediate response is expected, such as that following bronchodilator inhalation, the fall in Rint is more likely to be in excess of these limits in children with persistent isolated cough and in previously wheezy children. For between-occasion measurements, the limits of agreement are too wide for change in the individual to be judged with confidence, as is the case for between-occasion repeatability of FEV₁.