ORIGINAL ARTICLE

Interpretation of occupational peak flow records: level of agreement between expert clinicians and Oasys-2

D R Baldwin¹, P Gannon², P Bright², D T Newton², A Robertson³, K Venables⁴, B Graneek⁵, R D Barker⁶, A Cartier⁸, J-L Malo⁸, M Wilsher⁹, C F A Pantin⁷ and P S Burge²

¹ Department of Respiratory Medicine, Nottingham City Hospital, Nottingham, UK
² Department of Respiratory Medicine, Birmingham Heartlands Hospitals, Birmingham, UK
³ Occupational Health Department, Selly Oak Hospital, Birmingham, UK
⁴ Occupational Health Service, University of Oxford, Oxford, UK
⁵ Royal Marsden Hospital, London, UK
⁶ King’s Collage Hospital, London, UK
⁷ Department of Respiratory Medicine, North Staffordshire Hospital, UK
⁸ Hôpital du Sacre-Coeur, Montreal, Canada
⁹ Respiratory Services, Green Lane Hospital, Auckland, New Zealand

Correspondence to:
Correspondence to:
Dr D R Baldwin, Department of Respiratory Medicine, City Hospital, Nottingham, NG5 1PB, UK;
david.baldwin{at}nottingham.ac.uk

Background: Oasys-2 is a validated diagnostic aid for occupational asthma that interprets peak expiratory flow (PEF) records as well as generating summary plots. The system removes inconsistency in interpretation, which is important if there is limited agreement between experts. A study was undertaken to assess the level of agreement between expert clinicians interpreting serial PEF measurements in relation to work exposure and to compare the responses given by Oasys-2.

Method: 35 PEF records from workers under investigation for suspected occupational asthma were available for review. Records included details of nature of work, intercurrent illness, drug therapy, predicted PEF, rest periods, and holidays. Simple plots of PEF and the Oasys-2 generated plots were available. Experts were advised that approximately 1 hour was available to review the records. They were asked to score each work-rest-work (WRW) period and each rest-work-rest (RWR) period for evidence of occupational effect. At the end of each record scores of 0–100% were given for evidence of "asthma" and "occupational effect" for the whole record. Kappa values were calculated for each scored period and for the opinions on the whole record. The scores were converted into four groups (0–25%, 26–50%, 51–75%, 76–100%) and two groups (0–50% and 51–100%) for analysis. This is relevant to scores produced by Oasys-2. Agreement between Oasys-2 scores and each expert was calculated.

Results: 24 of 35 records were analysed by seven experts in the allotted time. For whole record occupational effect, median kappa values were 0.83 (range 0.56–0.94) for two groups and 0.62 (0.11–0.83) for four groups. For asthma, median kappa values were 0.58 (0–0.67) and 0.42 (0.15–0.70) for two and four groups respectively. For all WRW and RWR periods kappa values were 0.84 (0.42–0.94) and 0.70 (0.46–0.87) respectively. Agreement between Oasys-2 and individual experts showed a median kappa value of 0.75 (0.50–0.92) for two groups and 0.50 (0.39–0.70) for four groups. Kappa values for the median expert score v Oasys-2 were 0.75 for two groups and 0.67 for four groups. Agreement was poor for records with intermediate probability, as defined by Oasys-2.

Conclusion: Considerable variation in agreement was seen in expert interpretation of occupational PEF records which may lead to inconsistencies in diagnosis of occupational asthma. There is a need for an objective scoring system which removes human variability, such as that provided by Oasys-2.

Keywords: occupational asthma; peak expiratory flow; inter-observer variation; Oasys-2

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

• Moore, V. C., Cullinan, P., Sadhra, S., Burge, P. S. (2009). Peak expiratory flow analysis in workers exposed to detergent enzymes. Occup Med (Lond) 0: kqp083v1-kqp083 [Abstract] [Full Text]  
• Moore, V. C., Jaakkola, M. S., Burge, C. B. S. G., Pantin, C. F., Robertson, A. S., Vellore, A. D., Burge, P. S. (2009). PEF analysis requiring shorter records for occupational asthma diagnosis. Occup Med (Lond) 0: kqp081v1-kqp081 [Abstract] [Full Text]  
• Moore, V. C., Jaakkola, M. S., Burge, C. B. S. G., Robertson, A. S., Pantin, C. F. A., Vellore, A. D., Burge, P. S. (2009). A New Diagnostic Score for Occupational Asthma: The Area Between the Curves (ABC Score) of Peak Expiratory Flow on Days at and Away From Work. Chest 135: 307-314 [Abstract] [Full Text]  
• Tarlo, S. M., Balmes, J., Balkissoon, R., Beach, J., Beckett, W., Bernstein, D., Blanc, P. D., Brooks, S. M., Cowl, C. T., Daroowalla, F., Harber, P., Lemiere, C., Liss, G. M., Pacheco, K. A., Redlich, C. A., Rowe, B., Heitzer, J. (2008). Diagnosis and Management of Work-Related Asthma: American College of Chest Physicians Consensus Statement. Chest 134: 1S-41S [Abstract] [Full Text]  
• Fishwick, D, Barber, C M, Bradshaw, L M, Harris-Roberts, J, Francis, M, Naylor, S, Ayres, J, Burge, P S, Corne, J M, Cullinan, P, Frank, T L, Hendrick, D, Hoyle, J, Jaakkola, M, Newman-Taylor, A, Nicholson, P, Niven, R, Pickering, A, Rawbone, R, Stenton, C, Warburton, C J, Curran, A D, British Thoracic Society Standards of Care Subcomm, (2008). Standards of care for occupational asthma. Thorax 63: 240-250 [Full Text]  
• Chiry, S., Cartier, A., Malo, J.-L., Tarlo, S. M., Lemiere, C. (2007). Comparison of Peak Expiratory Flow Variability Between Workers With Work-Exacerbated Asthma and Occupational Asthma. Chest 132: 483-488 [Abstract] [Full Text]  
• Bolen, A. R, Henneberger, P. K, Liang, X., Sama, S. R, Preusse, P. A, Rosiello, R. A, Milton, D. K (2007). The validation of work-related self-reported asthma exacerbation. Occup. Environ. Med. 64: 343-348 [Abstract] [Full Text]  
• Fishwick, D., Bradshaw, L., Henson, M., Stenton, C., Hendrick, D., Burge, S., Niven, R., Warburton, C., Rogers, T., Rawbone, R., Cullinan, P., Barber, C., Pickering, T., Williams, N., Ayres, J., Curran, A. D (2007). Occupational asthma: an assessment of diagnostic agreement between physicians. Occup. Environ. Med. 64: 185-190 [Abstract] [Full Text]  
• Mapp, C. E., Boschetto, P., Maestrelli, P., Fabbri, L. M. (2005). Occupational Asthma. Am. J. Respir. Crit. Care Med. 172: 280-305 [Abstract] [Full Text]  
• Nicholson, P J, Cullinan, P, Newman Taylor, A J, Burge, P S, Boyle, C (2005). Evidence based guidelines for the prevention, identification, and management of occupational asthma. Occup. Environ. Med. 62: 290-299 [Abstract] [Full Text]  
• Gautrin, D., Newman-Taylor, A.J., Nordman, H., Malo, J-L. (2003). Controversies in epidemiology of occupational asthma. Eur Respir J 22: 551-559 [Abstract] [Full Text]  
• Beach, J., Hoffman, H. (2003). An Oasys for occupational asthma. CMAJ 169: 189-189 [Full Text]  
• (2003). Oasys-2 outperforms experts in diagnosing occupational asthma. Occup. Environ. Med. 60: 329-329 [Full Text]  
• Fishwick, D, Bradshaw, L M, Tate, P A, Curran, A D, Burge, P S (2003). Occupational asthma evaluation. Thorax 58: 461-461 [Full Text]  

eLetters:

Read all eLetters

Man versus machine

David Fishwick, et al.

Thorax Online, 1 Nov 2002 [Full text]

Response to David Fishwick

P Sherwood Burge, et al.

Thorax Online, 4 Dec 2002 [Full text]