Article Text
PDF
Abstract
Currently no tool exists to assess proceduralist skill at chest tube insertion. As inadequate doctor procedural competence has repeatedly been associated with adverse events, there is a need for a tool to assess procedural competence.
This study aims to develop and examine the validity of a tool to assess competency at insertion of a chest tube, using either the Seldinger technique or blunt dissection.
A 5-domain 100-point assessment tool was developed inline with British Thoracic Society guidelines and international consensus—the Chest Tube Insertion Competency Test (TUBE-iCOMPT). The instrument was used to assess chest tube insertion in mannequins and live patients. 29 participants (9 novices, 14 intermediate and 6 advanced) were tested by 2 blinded expert examiners on 2 occasions. The tool's validity was examined by demonstrating: (1) stratification of participants according to expected level of expertise (analysis of variance), and (2) test-retest and intertester reliability (intraclass correlation coefficient). The intraclass correlation coefficient of repeated scores for the Seldinger technique and blunt dissection, were 0.92 and 0.91, respectively, for test-retest results, and 0.98 and 0.95, respectively, for intertester results. Clear stratification of scores according to participant experience was seen (p<0.0001). There was no significant difference between scores obtained using mannequins or live patients. This study has validated the TUBE-iCOMPT, which could now be incorporated into chest tube insertion training programmes, providing a way to document acquisition of skill, guide individualised teaching, and assist with the assessment of the adequacy of clinician training.
- Pleural Disease
Statistics from Altmetric.com
Introduction
No tool currently exists to assess proceduralist skill at chest tube insertion. This is despite the fact that adverse events related to thoracentesis and chest tube insertion are common, and that inadequate doctor training has been identified as a key contributory factor.1 As trainees acquire skills at different rates, tools to assess when an individual is competent to perform a procedure are required.2
There are two different techniques used for insertion of a chest drain: Seldinger and blunt dissection. Although the practice of the Seldinger technique is increasing, it is not necessarily a safer or more efficacious method and both techniques are still used around the world.3 This study aims to develop and examine the validity of a tool to assess physician skill at insertion of a chest tube, using either the Seldinger or blunt dissection method.
Methods
Tool development
We developed an assessment tool that allows evaluation of chest tube insertion using either the Seldinger or blunt dissection technique—the Chest Tube Insertion Competency Test (TUBE-iCOMPT) (see online supplements). The test consists of five assessment domains: Domain 1, Preprocedural checks; Domain 2, Patient positioning and local anaesthetic; Domain 3, Blunt dissection skills; Domain 4, Seldinger skills; Domain 5, Suturing, drain connection and dressing. Scoring for each domain consists of a panel of objective check-box-style points and a more subjective global rating scale. To assess the Seldinger technique, Domains 1, 2, 4 and 5 are scored, while blunt dissection requires Domains 1, 2, 3 and 5. Testing of either technique gives a final score out of 100.
Testing protocol
The study was approved by the institutional ethics committee. Consenting participants were recruited from a tertiary teaching hospital in Brisbane between October 2011 and October 2013. Participants were classified according to previous experience as novice (no chest tube insertions), intermediate (attendance at a chest tube insertion workshop, but nil tube insertions in live patients), or advanced (more than 30 chest tube insertions in live patients).
Initially, the TUBE-iCOMPT was used to assess each subject inserting a chest tube in a mannequin (Super Annie http://www.simcentral.com.au), using blunt dissection (28Fr Argyle Trocar Catheter, Covidien, Mansfield, USA) and the Seldinger technique (16Fr Thal Quick Chest Tube, Cook Medical, Bloomington, USA), on two occasions 1–4 weeks apart. Two blinded expert examiners (interventional pulmonologists) and a senior respiratory trainee scored each performance either in real time or at a later date from a video recording.
Following data collection using mannequins, the TUBE-iCOMPT was applied to assess a group of intermediate and advanced subjects inserting a chest tube into live patients. Intermediates were closely supervised. If they required prompting or correction of technique, they were scored zero for the corresponding point on the TUBE-iCOMPT.
Statistical analysis
Data within groups was examined for normality and found to be well approximated by a normal distribution. To examine test validity, the reproducibility of scores and the instrument's ability to discriminate between subjects of varying experience were assessed. The reproducibility of test-retest scores (occasion 1 vs occasion 2) and inter-tester scores (pulmonologist 1 vs pulmonologist 2 vs advanced trainee) were gauged on results derived from mannequin testing by calculating the intraclass correlation coefficient (ICC). The tool's ability to discriminate between subjects of varying experience was assessed by comparing groups with analysis of variance (ANOVA). As an indication of the correlation between results from live patient and mannequin testing, scores corresponding to each group were plotted against one another, and significant differences between the two assessed with Student t test. Statistical analysis was performed with STATA software V.12 (StataCorp, College Station, Texas, USA).
Results
Consenting participants numbering 29 were recruited for mannequin testing of the Seldinger and blunt dissection techniques (9 novices, 14 intermediates and 6 advanced) and 12 different participants for live patient testing (4 advanced Seldinger technique, 6 intermediate Seldinger technique, 1 advanced blunt dissection, 2 intermediate blunt dissection).
As can be seen in figure 1, the tool accurately stratified participants according to experience with non-overlapping 95% CIs, when used to assess the Seldinger technique and blunt dissection. The mean scores (95% CI) derived from mannequin testing for novices, intermediates and advanced were 42.2 (39.5 to 45.0), 71.3 (69.4 to 73.1) and 87.6 (86.5 to 88.7), respectively, for the Seldinger technique, and 48.6 (45.7 to 51.5), 74.3 (72.6 to 75.9) and 87.0 (85.7 to 88.4), respectively, for blunt dissection. Groups were all significantly different at the p<0.0001 level (ANOVA). These groups remained distinct when testing was performed using live patients, and the results were not significantly different to those obtained from mannequin testing (intermediates p=0.56, advanced p=0.10).
Test scores by group for Seldinger technique and blunt dissection. The columns represent mean scores and error bars 95% CI.
Differences between groups of varying experience remained highly significant even when each test domain was examined in isolation (p<0.005) except for domain 1 where there was no significant difference between adjacent groups (ie novice vs intermediate, intermediate vs advanced) (p=0.07–0.5).
Results from mannequin testing showed robust reproducibility. For intertester data, the ICC between the expert examiners was 0.98/0.95 (Seldinger/blunt dissection), while that between the expert examiners and the advanced trainee was 0.96/0.95 (Seldinger/blunt dissection technique). For test-retest data, the ICC was 0.92/0.91 (Seldinger/blunt dissection).
Discussion
This study has validated a new instrument to assess skill at insertion of a chest tube, using either blunt dissection or the Seldinger method. The test accurately stratifies participants according to skill level and is reliable, giving highly reproducible results on repeat testing, when marked by examiners of differing experience or when the procedure is performed on live patients or mannequins. The instrument could now be incorporated into chest tube insertion training programmes, providing a way to document the acquisition of skill, guide individualised teaching and assist in the assessment of the adequacy of clinician training.
Although there are numerous facets to what constitutes the validity of a new test, there is general consensus that for a new evaluative tool to be clinically valid, it must be reproducible and measure what it purports to measure, in this case, procedural competence.2 Although it is the repetitive practice of a technique that leads to competence, the number of procedures performed is not in itself a reliable indicator of competence. For this reason, a number of different aspects of validity were addressed in the design of the TUBE-iCOMPT. The assessment points in the instrument were constructed from international consensus of over 100 health professionals, including pulmonologists, intensive care specialists and cardiothoracic surgeons, on the essential steps of correct chest tube insertion.4 Weighting for the individual assessment points was assigned in line with British Thoracic Society Guidelines.5 A global rating scale was included to address the intangible components of procedural competence gained with time. Finally, there was a large separation between groups of varying experience (novice, intermediate, advanced), to maximise the (imperfect) delineation between levels of expected competence.
A major strength of the TUBE-iCOMPT is its flexibility. The five scoring domains, each comprised of a combination of check-list-style marking points and a global rating scale, allow the instrument to assess the generic skills required for the safe insertion of any needle or tube into the pleural space, while remaining flexible enough to assess either blunt dissection or the Seldinger method. It should be noted however, that procedural competence does not necessarily equate to overall clinical competence, which also requires other skills, such as clinical judgement, as to when and what pleural procedure should be performed.
This study has not attempted to answer the question of what score is required to indicate a particular level of competence, such as when a trainee is ready to perform unsupervised procedures on a patient. To define this relationship, further study is required. However, the score obtained with the assessment tool should not be seen as an end unto itself, but rather as a useful addition to a comprehensive teaching programme, which helps the trainee recognise specific deficiencies in their technique while allowing the trainer to follow the student's acquisition of skills and appropriately tailor further instruction.2
One possible limitation to this study is that a large amount of the data was derived from applying the TUBE-iCOMPT to assess procedures performed on a mannequin. We believe this was justified to establish the reproducibility of the instrument, as repeated testing needed to be made during a procedure with standardised complexity (which is not the case with live patient procedures). Following establishment of the test's reproducibility using mannequins, the performance of the tool when used with live patients was assessed. Further, we feel that it is important the TUBE-iCOMPT can discriminate between operators of different skill level and identify specific deficiencies in technique, regardless of whether the procedure is performed with mannequins or live patients, as initial training in simulated clinical environments is rapidly becoming the standard for procedural training.
In conclusion, this TUBE-iCOMPT constitutes a useful contribution to the growing number of instruments available to assess the procedural skill of pulmonologists. This study has validated the TUBE-iCOMPT, which could now be incorporated into chest tube insertion training programmes, providing a way to document the acquisition of skill, guide individualised teaching and assist in the assessment of the adequacy of clinician training.
Acknowledgments
The primary author acknowledges support from The University of Queensland and The Royal Brisbane and Women's Hospital Foundation. The E-learning course on chest drain insertion is freely available worldwide.4
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
- Data supplement 2 - Online supplement
- Data supplement 3 - Online supplement
Footnotes
-
Contributors MRS 60%: Wrote the assessment instrument, designed the study, recruited and (Guarantor) tested the participants, analysed the data and wrote the manuscript. FB 10%: Expert assessment of participants. AJR 5%: Advanced registrar assessment of participants. HEW 5%: Assisted in writing of the assessment instrument and manuscript. DIKF 20%: Assisted in writing of the assessment instrument and study formulation, expert assessment of participants, assisted with the manuscript preparation.
-
Competing interests None.
-
Patient consent Obtained.
-
Ethics approval The Royal Brisbane and Women's Hospital Human Research Ethics Committee.
-
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- Airwaves