You are here

Article Text

Article menu
Tuberculosis
Original article
The prospective evaluation of the TB strain typing service in England: a mixed methods study
Free
  1. J Mears1,
  2. E Vynnycky2,3,
  3. J Lord4,
  4. M W Borgdorff5,6,
  5. T Cohen7,
  6. D Crisp8,
  7. J A Innes9,
  8. M Lilley10,
  9. H Maguire11,
  10. T D McHugh12,
  11. G Woltmann13,
  12. I Abubakar1,14,
  13. P Sonnenberg1
  1. 1Research Department of Infection and Population Health, University College London, London, UK
  2. 2Modelling and Economics Unit, Public Health England, London, UK
  3. 3Department of Infectious Disease Epidemiology, London School of Hygiene and Tropical Medicine, London, UK
  4. 4Health Economics Research Group, Brunel University, London, UK
  5. 5Department of Infectious Diseases, Public Health Service (GGD) Amsterdam, Amsterdam, The Netherlands
  6. 6Department of Clinical Epidemiology, Academic Medical Centre, University of Amsterdam, The Netherlands
  7. 7Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, USA
  8. 8George Elliot NHS Trust, Warwickshire, UK
  9. 9Department of Infection and Tropical Medicine, Birmingham Heartlands Hospital, Heart of England NHS Foundation Trust, Birmingham, UK
  10. 10South Midlands and Hertfordshire Public Health England Centre, Herts, UK
  11. 11Field Epidemiology Services, Public Health England, London, UK
  12. 12Research Department of Infection, University College London, London, UK
  13. 13Department of Respiratory Medicine, University Hospitals Leicester, Glenfield Hospital, Leicester
  14. 14Clinical Trials Unit, Medical Research Council, London, UK
  1. Correspondence to Dr Pam Sonnenberg, Research Department of Infection and Population Health, University College London, Mortimer Market Centre, London WC1E 6JB, UK; p.sonnenberg{at}ucl.ac.uk

Abstract

Background In response to rising TB notification rates in England, universal strain typing was introduced in 2010. We evaluated the acceptability, effectiveness and cost-effectiveness of the TB strain typing service (TB-STS).

Methods We conducted a mixed-methods evaluation using routine laboratory, clinic and public health data. We estimated the effect of the TB-STS on detection of false positive Mycobacterium tuberculosis diagnoses (2010–2012); contact tracing yield (number of infections or active disease per pulmonary TB case); and diagnostic delay. We developed a deterministic age-structured compartmental model to explore the effectiveness of the TB-STS, which informed a cost-effectiveness analysis.

Results Semi-structured interviews explored user experience. Strain typing identified 17 additional false positive diagnoses. The TB-STS had no significant effect on contact tracing yield or diagnostic delay. Mathematical modelling suggested increasing the proportion of infections detected would have little value in reducing TB incidence in the white UK-born population. However, in the non-white UK-born and non-UK-born populations, over 20 years, if detection of latent infection increases from 3% to 13% per year, then TB incidence would decrease by 11%; reducing diagnostic delay by one week could lead to 25% reduction in incidence. The current TB-STS was not predicted to be cost-effective over 20 years (£95 628/quality-adjusted life-years). Interviews found people had mixed experiences, but identified broader benefits, of the TB-STS.

Conclusions To reduce costs, improve efficiency and increase effectiveness, we recommend changes to the TB-STS, including discontinuing routine cluster investigations and focusing on reducing diagnostic delay across the TB programme. This evaluation of a complex intervention informs the future of strain typing in the era of rapidly advancing technologies.

  • Tuberculosis

https://doi.org/10.1136/thoraxjnl-2014-206480

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Key messages

    What is the key question?

    • Is the TB strain-typing service (TB-STS) in England an effective or cost-effective public health intervention?

    What is the bottom line?

    • The TB-STS, which includes universal mycobacterial interspersed repetitive units-variable number tandem repeats (MIRU-VNTR) typing from each TB case and cluster investigations, was not found to be effective or cost-effective in its current form; however, broader benefits for TB control and research support the continuation of the service, though with some significant changes.

    Why read on?

    • In the context of the rapid development of typing methodologies (eg, whole genome sequencing), political commitment to genomic analysis1 and the development of Public Health England's 2014–2019 National TB Strategy,2 this evaluation provides important evidence for policymakers and those working in TB control.

    Introduction

    Molecular typing of Mycobacterium tuberculosis is a tool for TB surveillance and control. Following increases in TB notification rates in the UK, the Chief Medical Officer recommended the introduction of a national TB strain typing service (TB-STS),3 which was implemented across England in 2010. The TB-STS is a complex public health intervention involving multiple interacting components (laboratory, public health and clinical services). The first M. tuberculosis isolate from every culture-confirmed TB patient is typed using 24 loci mycobacterial interspersed repetitive units-variable number tandem repeats (MIRU-VNTR), a standardised molecular typing method.4 Two or more patients are grouped into strain typing ‘clusters’ if they are diagnosed within two years and have indistinguishable MIRU-VNTR strain types (with at least 22 loci). Where specified criteria are met,5 cluster investigations are launched to try to establish epidemiological links between clustered patients, thereby identifying the transmission setting and/or an outbreak. A full description of the TB-STS, laboratory guidelines for MIRU-VNTR strain typing and reporting6 and a handbook for public health actions, including criteria for cluster investigations,5 are available.

    It was hypothesised that prospective strain typing could be used in real time to inform public health action. Strain typing could help to identify individuals that are misdiagnosed as having TB due to processing error or contamination from clinic to laboratory. Cluster investigations could result in better-targeted contact tracing, increasing the detection of recently infected individuals who may benefit from preventive therapy, and accelerating the diagnosis of active cases.

    An evaluation of the TB-STS was commissioned alongside the roll-out of the service, providing an opportunity to conduct the first prospective evaluation of a national TB-STS. We present an evaluation of the acceptability, effectiveness and cost-effectiveness of the TB-STS in England.

    Methods

    We conducted a mixed-methods evaluation7 using routine laboratory, clinic and public health data and primary data collection through semi-structured interviews (table 1). We investigated the occurrence of false positive isolates, the contact tracing yield and diagnostic delay. These parameters, together with data obtained from initial and follow-up cross-sectional surveys evaluating user perception and implementation of the TB-STS,8 were used to develop a mathematical model of effectiveness of the TB-STS, which informed a model of cost-effectiveness (figure 1).

    View this table:
    Table 1

    Median contact tracing yield for pulmonary index cases by clustering and whether the cluster was investigated

    Figure 1
    Figure 1

    Components of the mixed-methods evaluation. The arrows show how the different components of the evaluation were used to inform the semi-structured interviews and parameters for the transmission and cost-effectiveness models. Detailed results from the initial and follow-up surveys are presented elsewhere.8 TB-STS, TB strain typing service.

    False positive detection

    We hypothesised that the TB-STS would identify previously unknown incidents of false positive TB isolation, thereby reducing the cost of unnecessary treatment and public health action. Isolates from the three TB reference laboratories were queried if they had consecutive source or reference laboratory identification numbers, or were typed within seven days of each other and had matching 24 MIRU-VNTR profiles. Data on the number of incidents queried and their outcomes were collected. An email survey to source laboratories established whether the incidents were already known to them and whether any patients had been treated unnecessarily.

    Contact tracing yield

    We hypothesised that the number of infections and TB cases identified through contact tracing for each index case (termed contact tracing yield) would be greater in cases that were part of investigated clusters compared with cases in clusters that were not investigated. Median contact tracing yield for pulmonary index cases, by clustering and whether the cluster was investigated, were calculated using data from the North Central London (NCL) sector and the Leicester TB service, and included index cases diagnosed in 2011. We did a sensitivity analysis that assumed that index cases with missing contact tracing information yielded no cases of active disease or latent TB infection (LTBI) (as those with positive results would be more likely to be recorded).

    Diagnostic delay

    We hypothesised that the diagnostic delay (the time between symptom onset and case notification) would be reduced following the introduction of the TB-STS because cluster investigations would lead to undiagnosed TB cases being actively identified earlier. Using all pulmonary TB cases diagnosed in England during 2011 that were part of clusters that were investigated, we compared the median diagnostic delay in cases that were diagnosed after a cluster investigation had started to cases that were diagnosed before the cluster was investigated. The first two cases in each cluster were excluded to remove clusters of two and to take into account possible household transmission.

    Data were analysed using Stata V.12. We present medians and IQRs. The Wilcoxon–Mann–Whitney test was used to compare differences in contact tracing yield and diagnostic delay.

    Effectiveness of the TB-STS

    We used a deterministic age-structured model to explore the possible reductions in TB incidence as a result of the TB-STS over a 20-year period (figure 2). This extends previous models considering the transmission dynamics of M. tuberculosis in England and Wales9 and recent work on preventive therapy.10 Details of the model are provided in online supplementary file 1A. The model incorporates contact between individuals and rates of immigration and emigration based on Office for National Statistics data.11 For simplicity, the model considers only pulmonary TB and considers three different epidemiological scenarios—low, medium and high incidence—comparable to that in the white UK-born population (with decreasing annual risk of infection (ARI)), non-white UK born (0.1% ARI) and the non-UK born (1% ARI).

    Figure 2
    Figure 2

    General structure of the transmission model. PT, preventive treatment. Coloured text and shading is used to reflect similar categories of people: yellow shading is used for people on PT, purple text is used for compartments for people with latent infection, green text is used for newly infected or reinfected people, red is used for diseased people and orange, blue and pink text is used for detected cases, people on treatment and the recovered respectively.

    In the absence of the TB-STS, the average diagnostic delay is assumed to be that estimated for cases that were not in clusters. We explore the effect of a one week reduction in diagnostic delay due to the TB-STS and assume that patients start TB treatment (on average) two weeks after diagnosis.

    In the absence of the TB-STS, 3% of all (recently or latently) infected individuals are assumed to have been detected each year. This proportion is unknown, but was probably low, given the low number of latently infected contacts per index case (see online supplementary file 1). We explore the effect of assuming that it increased to 13% after the introduction of the TB-STS. We assume that uptake of preventive treatment (PT) among those eligible is 95% and 85% complete the course of treatment. In sensitivity analyses, we explore the effect of pessimistic and optimistic assumptions about uptake (30% and 100%, respectively) and completion (50% and 100%, respectively) of PT.

    Cost-effectiveness of the TB-STS

    We aimed to estimate the cost-effectiveness of the TB-STS as an addition to the current system for TB control in England. The cost-effectiveness analysis is illustrated in figure 3, and a full description of the methods and results is given in online supplementary file 1B. The analysis adopted a public sector perspective. Estimates of the cost of setting up and operating the national TB-STS were made based on information from Public Health England (PHE) and the TB Reference Laboratories. Capital costs were annuitised over an assumed 10-year lifetime for equipment. Costs and health effects (in quality-adjusted life-years (QALYs)) were estimated over a 20-year time horizon, and applying a 3.5% annual discount rate to both costs and QALYs (as recommended by the National Institute of Health and Care Excellence (NICE)).12 The results of the transmission model provided estimates over this 20-year period for each modelled scenario of the number of contacts with latent infection identified, the number of contacts starting preventative treatment, the number of people with active TB diagnosed and starting treatment, as well as the impact on the number of incident TB cases. The costs associated with diagnosis and the treatment of latent and active TB were estimated based on recommended practice and expert opinion. The unit costs of tests, medications, outpatient contacts and inpatient stays were obtained from national sources.13 ,14 The QALY effects of TB were estimated based on case-fatality rates15 and life expectancy by age of TB incidence, and estimates of the duration and utility loss associated with active disease.16 We applied the NICE threshold of £20 000–30 000.12

    Figure 3
    Figure 3

    Structure of the cost-effectiveness analysis. QALY, quality-adjusted life-year; TB-STS, TB strain typing service.

    User experience

    We conducted semi-structured telephone interviews with the TB-STS leads at 24 (of 26) local health protection units (HPUs) in December 2012, following piloting in the other two HPUs. These explored how strain typing information was used by different HPUs and the variation in user experience. Using a thematic analysis, JM extracted the data, coded and categorised it into the themes and subthemes identified across the interviews.

    Results

    False positive detection

    Between June 2010 and June 2012, 11 059 TB isolates were typed at the reference laboratories. There were 70 suspected incidents of false positive TB isolation (0.6%), of which 30 (42.9%) were confirmed as false positive, giving a rate of false positive TB isolation in England of 0.3%. Seventeen (56.7%) of the suspected incidents were not known to the source laboratories, and 8 patients were started on unnecessary treatment.

    Contact tracing yield

    In 2011, for NCL sector and Leicester (table 1), the median number of contacts screened, with active TB and LTBI, was significantly greater in clustered compared with unique cases. No difference was observed between cases that were part of a cluster that was investigated compared with not investigated (p=0.474, 0.896 and 0.330, respectively, for contacts screened, with active disease or LTBI). Sensitivity analysis that assumed that index cases with missing contact tracing information yielded no cases of active disease or LTBI found that the comparison of median yields in unique and clustered cases remained the same, but were no longer significant (p=0.06 and 0.418 for contacts with active TB and LTBI, respectively). Median yields were broadly similar for cases in clusters investigated or not (p=0.872, 0.819 and 0.436, respectively, for contacts screened, with active disease or LTBI).

    Diagnostic delay

    We identified 318 pulmonary TB cases that were in a cluster that was investigated, of which 238 had information on diagnostic delay (table 2). The median diagnostic delay was not significantly different in cases that were in a cluster that was not investigated (n=139; 62 days) or diagnosed before a cluster investigation was launched (n=117; 85 days) compared with those diagnosed after the start of the cluster investigation (n=121; 77 days).

    View this table:
    Table 2

    The diagnostic delay* for clustered pulmonary cases from 2011, based on whether they were in a cluster that was investigated, and whether they were diagnosed before or after the investigation was launched

    Effectiveness of the TB-STS

    Figure 4 summarises the predictions of the impact of the TB-STS on TB incidence for the three epidemiological scenarios considered. For the white UK-born population, the predicted incidence decreased from 4 to <1 per 100 000 per year between 2005 and 2030 in the absence of additional interventions. Reducing diagnostic delay and/or increasing the proportion of infections that were detected in this population because of the TB-STS is predicted to have little impact on TB incidence (figure 4A).

    Figure 4
    Figure 4

    Predicted impact of the TB strain typing service (TB-STS). Predicted impact of reducing diagnostic delay from 12 to 11 weeks and increasing the percentage of infections that are detected through the TB-STS from 3% to 13%. The number of cases occurring per 100 000 per year in a setting in which the TB incidence is (A) similar to that in the white UK-born population (declining annual risk of infection (ARI)); (B) similar to that in the non-white UK-born population group (ARI=0.1% per year); (C) similar to that in a high transmission, non-UK-born population group (ARI=1% per year); and (D) the average annual number of cases prevented per 100 000 population for these scenarios over 20 years after the introduction of the TB-STS.

    For populations with a similar incidence to that of the non-white UK-born population, increasing the proportion of infections detected from 3% per year to 13% is predicted to lead to a small reduction in TB incidence compared with that in the absence of the TB-STS. For example, for this scenario, 20 years after the introduction of the TB-STS, the predicted incidence is 11% less than that in the absence of TB-STS (figure 4B) (ranging between 4% and 12% for pessimistic and optimistic assumptions, respectively, relating to uptake and completion of PT), with about one case prevented per 100 000 per year over this period (figure 4B). Combining an increase in the proportion of infections detected to 13% per year with a one week reduction in diagnostic delay is predicted to approximately double the reduction in TB incidence, with just over two cases prevented per 100 000 per year over the period 2010–2030 (figure 4D).

    Of the scenarios considered, the TB-STS is predicted to lead to the greatest reductions in TB incidence in high transmission settings, similar to the non-UK-born population. For this scenario, 20 years after the introduction of the TB-STS, the predicted incidence is about 15% less than that in the absence of the TB-STS (figure 4C), with about 10 cases prevented per 100 000 per year over this period (figure 4C). When an increase in the proportion of infections detected to 13% per year is combined with a one week reduction in diagnostic delay, the reduction in TB incidence increases to 40% (figure 4C), with >30 cases prevented per 100 000 per year over the period 2010–2030 (figure 4D).

    Cost-effectiveness of the TB-STS

    The cost of implementing and running the TB-STS was estimated at approximately £1 m per year. Assuming the TB-STS leads to a small improvement in the contact tracing yield in a medium-incidence population (similar incidence to that of the non-white UK-born population), the service was not estimated to be cost-effective over a 20-year period: £95 628 per QALY if the proportion of LTBI detected were to increase from 3% to 4%. Assuming an increase from 3% to 13%, the estimated incremental cost per QALY gained (£54 539) still did not reach a level considered to be cost-effective in the UK. These results were much more sensitive to reductions in the diagnostic delay: for example, if the TB-STS reduced diagnostic delay by one week, the system would save £85 million and gain over 16 000 QALYs over 20 years.

    User experience

    The main themes that emerged from the semi-structured interviews are shown in box 1. The interviews demonstrated a wide variation in user engagement with the TB-STS, use of the strain typing information and experience of the service. Many strain typing leads felt that the potential added value of strain typing had not been realised with the current service; however, strain typing should not be stopped.

    Box 1

    User experience of the TB-STS. Main themes that emerged from the semi-structured interviews with health protection staff

    • Variation in processes. The way the strain typing information was used and acted on at the local level was dependent on context, for example, the availability of resources and local priorities. Importantly, no additional funds were allocated for local health protection teams to act on the information provided by the TB-STS. Some areas never used the information, in other areas it had been integrated into normal practice.

      • “We have the luxury to spend more time to have a quick look if they are potentially linked.”

      • “We have done quite a bit of work getting a standardised form for TB nurses to use...they’ve adapted the questionnaire but I cannot give you assurance that it is being implemented properly. In my area the service doesn’t see it as a priority.”

      • “I have some concern on … the full cluster investigation—in the good will of our NHS partners and how much they contribute to that. There are mutterings about not being commissioned to do them [the investigations]”.

    • Uses of the TB-STS. The TB-STS was reported to be useful for confirming/refuting suspected transmission, informing case definitions in ongoing outbreaks, helping guide contact tracing, and monitoring and evaluating local TB services. However, many people reported that the information arrived too late, it generated more work and created confusion.

      • “By the time we get the report we already know about it and have carried out actions. It's always too late… [the service] is not generating new information.”

    • Impact of the TB-STS. Many respondents felt that the TB-STS had not yet delivered a public health benefit and that it generated more work that led to little benefit. The TB-STS added value in other more general ways that strengthen the TB service as a whole, such as engaging TB nurses with the public health aspects of TB. Despite the lack of outcomes and the drain on resources, people felt that it would be ‘regressive’ to stop the STS.

      • “Hasn’t added benefits so far, but hasn’t highlighted anything we weren’t already dealing with...Very happy to receive the strain typing—wouldn't want to not receive it.”

      • “It’s a good thing. It’s been helpful for us. It’s a good way of getting people more interested in more work around TB. Doing cluster investigations has been a good way also of building up relations with the nurses.”

    Discussion

    This is the first mixed-methods prospective evaluation of a national TB-STS, a complex intervention, and informs the future of such services in the era of rapidly advancing typing technologies. The evaluation approach can be applied to any future typing method. The TB-STS, which includes universal MIRU-VNTR typing from each TB case and cluster investigations, was not found to be effective or cost-effective in its current form. Mathematical modelling to explore the potential impact of increasing the proportion of infections detected and decreasing diagnostic delay suggests that the TB-STS would have little value in reducing TB incidence in low-incidence settings (similar to that in the white UK-born population). Of the scenarios explored, the greatest potential impact is estimated in high-incidence settings, if the proportion of LTBI detected increases to 13% and diagnostic delay reduces by one week. According to the NICE threshold of £30 000 and assuming that (at most) the TB-STS would achieve only a modest increase in latent infections identified, the service was not predicted to be cost-effective over a 20-year period (£95 628 per QALY). However, even a small reduction in time to diagnosis of active cases has the potential to lead to a large decline in TB incidence over 20 years in some populations, leading to large cost savings. This information must be weighed up with the broader benefits of the service described in box 2 and insights captured through semi-structured interviews (such as the increased engagement with TB control at the local level and standardisation of contact tracing practices (box 1)). To reduce costs, improve efficiency and increase effectiveness, we have recommended continuation of the TB-STS but with significant changes to the service (see online supplementary file 2). These include discontinuation of routine cluster investigations and a focus on reducing diagnostic delay in the TB programme as a whole, of which the TB-STS is one part.

    Box 2

    Examples of the wider benefits of a national TB strain typing service (TB-STS)

    • To understand the national and local epidemiology of TB: TB strain typing in New York has enabled molecular epidemiological analyses to contribute to understanding of the TB epidemic and they have been able to tailor their public health response, especially among those with HIV or multidrug-resistant TB (MDR-TB).28 National strain typing in the Netherlands has contributed greatly to their understanding of TB epidemiology.29

    • To understand the molecular epidemiology of TB, thereby contributing to the global knowledge of TB.26

    • To monitor and evaluate TB programmes: an outreach screening service in the homeless and drug-using population in Rotterdam was evaluated using strain typing;22 long-term trends are used to evaluate TB control strategies, for example, in San Francisco and the Netherlands.29

    • To meet international obligations for molecular surveillance: European Centre for Disease Prevention and Control Molecular surveillance of MDR-TB in Europe project.30

    • To create a national repository of strain types: this can be used for national and local analyses, larger research projects, and provides the opportunity for national and international collaboration.

    Complex public health interventions, especially when these are at a large-scale level, are often implemented in a way that makes rigorous evaluation a challenge.17 There were no adequate control data from before the intervention, and the national roll-out of the TB-STS made a cluster-randomised controlled study design impossible. Parallels can be drawn between the TB-STS and the National Chlamydia Screening Programme where assessing the success of the programme has been difficult.18

    The lack of observed impact of the TB-STS may be due to (1) a true lack of impact, (2) the inability to observe the impact within the observation period or (3) the (limited) implementation of the service. Firstly, there may have been no impact, even if the TB-STS had been implemented perfectly because the 2-year evaluation period was relatively short to observe an impact. Strain typing information may not reach TB service staff early enough to inform contact screening decisions in a meaningful way (box 1). Possible reasons for this include the time needed to produce a typing result and/or the lack of sufficiently sophisticated reporting software. Furthermore, inadequate resources to act may be contributing—the generation of the strain typing information was well-resourced but no funds were allocated to local TB teams to embed the information in their decision processes (box 1). Secondly, we are evaluating the marginal impact of the TB-STS in a setting where a TB control programme already exists (which includes traditional TB control strategies, such as stone in the pond contact tracing).19 Measuring the impact of the TB-STS in isolation may therefore not be feasible. Thirdly, there was limited implementation of the service due to delayed development of clustering software as part of a integrated TB management system to capture linked cases, contacts and strains; and limited resources for local public health action. This also influenced the ability to evaluate the TB-STS, where suboptimal data collection systems meant that although some primary data collection was conducted, the evaluation was reliant on routine data sources to estimate model parameters.

    The findings of this study are important given the current trend to introduce and upgrade national typing services.20 ,21 The literature shows the value of TB typing in specific populations22 and for outbreaks,23 ,24 but there are few studies that look at the public health value of nationwide TB strain typing and the subsequent cluster investigations (box 3). Consistent with our findings, evidence from the Netherlands suggests that one may not expect to see an effect of a national TB-STS on contact tracing yield.25 The added strength of our study is the use of these outputs to predictively model the effect of the TB-STS on transmission over a 20-year period and to explore cost-effectiveness. While the broader benefits of the TB-STS cannot be quantified, the importance of the service for research and surveillance is acknowledged in box 2. Put simply, the benefits are twofold. Firstly, there is the value of a national dataset combining clinical, epidemiological and molecular information for each TB patient. This will lead to the possibility of multiple future analyses, TB programme evaluations and research projects. Secondly, the outcomes of such projects will lead to the even broader benefits of increased understanding about TB epidemiology, lineage, transmission and control. These benefits, however, may not require prospective strain typing, but could be gained through retrospective strain typing.

    Box 3

    ‘Research in context’

    Systematic review

    Universal national TB strain typing services (TB-STS) have been implemented in various countries including the USA, the Netherlands, Denmark, Slovenia and England, and more countries are planning such services. We searched PubMed for articles describing national TB-STS using the terms “tuberculosis” and “strain typing” or “genotyping” or “fingerprinting” or “miru vntr” or “rflp” and “routine” or “service” or “universal” or “nationwide”.

    We conducted an additional search in PubMed using the search terms “tuberculosis” and “strain typing” or “genotyping” or “fingerprinting” or “miru vntr” or “rflp” and “cost-effectiveness”. Although universal typing services have contributed greatly to the local and international understanding of TB epidemiology,26 no service-wide evaluations of their effectiveness and cost-effectiveness have been conducted. This is often the case with complex public health interventions.

    Despite the lack of evidence for the effectiveness of such a service, there is a political and scientific momentum to upgrade the current services based on mycobacterial interspersed repetitive units-variable number tandem repeats typing to whole genome sequencing.1 ,27

    Interpretation

    Based on the available quantitative data, there was no evidence to suggest that the current TB-STS in England is an effective or cost-effective public health intervention (this finding can be applied to the TB-STS, whatever the typing methodology applied); however, the broader benefits for TB control and research at a modest cost support the continuation of the service following recommended changes.

    Public health interventions are rarely evaluated despite mixed-method evaluations being very informative. Decision-makers planning and implementing complex public health interventions should ensure the collection of good quality data for the prospective evaluation of such interventions and be responsive to the findings in order that public funds are allocated effectively.

    In the context of the rapid development of typing methodologies (eg, whole genome sequencing), political commitment to genomic analysis1 and the development of PHE's 2014–2019 National TB Strategy,2 this evaluation provides important evidence for policymakers. Unless current or new typing and diagnostic techniques accelerate diagnosis (including through analysis of primary specimens), reduce diagnostic delay, dramatically reduce the time it takes to type and/or are embedded in a user-friendly standardised TB management system, the adoption of such a method alone is unlikely to impact on TB control. Comprehensive TB control strategies that aim to reduce TB incidence over the next decades need ongoing evaluation of proposed interventions. This includes evaluation of effective public health responses and appropriate use of strain typing, clinical and epidemiological information.

    Acknowledgments

    PHE Mycobacterial Reference Laboratories: National Mycobacterial Reference Laboratory, Birmingham Reference Laboratory and Newcastle Reference Laboratory for providing the false positive TB isolation data. TB Section, Centre for Infectious Disease Surveillance and Control, PHE for contributing the public health data. Sue Dart and Jacqui White for contributing contact tracing data from the North Central London Sector.

    References

    1. ↵
      Department of Health. 100,000 Genomes Project [Internet] (cited 4 June 2014). http://www.genomicsengland.co.uk/100k-genome-project
    2. ↵
      Public Health England. National TB Strategy 2014–2019 Consultation [Internet]. (cited 1 Jul 2014). http://www.hpa.org.uk/Publications/InfectiousDiseases/Tuberculosis/1403TBstrategyconsultation2014/
    3. ↵
      Department of HealthDonaldson SL. Stopping tuberculosis in England: an action plan from the Chief Medical Officer [Internet]. 2004 (cited 11 Apr 2012). http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4090417
    4. ↵
      1. Supply P,
      2. Allix C,
      3. Lesjean S, et al
      . Proposal for standardization of optimized mycobacterial interspersed repetitive unit-variable-number tandem repeat typing of Mycobacterium tuberculosis. J Clin Microbiol 2006;44:4498–510. doi:10.1128/JCM.01392-06
      OpenUrlAbstract/FREE Full Text
    5. ↵
      Public Health England. TB Strain Typing and Cluster Investigation Handbook 3rd Edition [Internet]. 2014 (cited 4 Jun 2014). http://www.hpa.org.uk/webc/HPAwebFile/HPAweb_C/1317140774833
    6. ↵
      1. Brown T,
      2. Evans JT,
      3. Sails AD, et al
      . HPA Mycobacterium tuberculosis Strain Typing: a guide to data production and distribution. HPA, 2012.
    7. ↵
      1. Campbell M,
      2. Fitzpatrick R,
      3. Haines A, et al
      . Framework for design and evaluation of complex interventions to improve health. BMJ 2000;321:694–6. doi:10.1136/bmj.321.7262.694
      OpenUrlFREE Full Text
    8. ↵
      1. Mears J,
      2. Abubakar I,
      3. Crisp D, et al
      . Prospective evaluation of a complex public health intervention: lessons from an initial and follow-up cross-sectional survey of the Tuberculosis Strain Typing Service in England. BMC Public Health 2014;14:1023. doi:10.1186/1471-2458-14-1023
      OpenUrlCrossRefPubMed
    9. ↵
      1. Vynnycky E,
      2. Fine PE
      . The natural history of tuberculosis: the implications of age-dependent risks of disease and the role of reinfection. Epidemiol Infect 1997;119:183–201. doi:10.1017/S0950268897007917
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Vynnycky E,
      2. Sumner T,
      3. Fielding KL, et al
      . Tuberculosis control on South African goldmines: mathematical modelling of a trial of community-wide isoniazid preventive therapy. Am J Epidemiol, in press.
    11. ↵
      Office for National Statistics. Migration indicators tool. ONS, 2010.
    12. ↵
      National Institute of Health and Care Excellence. Methods for development of NICE public health guidance [Internet]. 2009 (cited 14 Dec 2011). http://www.nice.org.uk/phmethods2009
    13. ↵
      Department of Health. NHS Reference Costs 2010–11 Collection Guidance [Internet]. 2010 (cited 30 Nov 2011). http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_122803
    14. ↵
      PSSRU|Unit Costs of Health and Social Care 2011 [Internet]. (cited 13 Apr 2012). http://www.pssru.ac.uk/project-pages/unit-costs/2011/index.php
    15. ↵
      1. Crofts JP,
      2. Pebody R,
      3. Grant A, et al
      . Estimating tuberculosis case mortality in England and Wales, 2001–2002. Int J Tuberc Lung Dis 2008;12: 308–13.
      OpenUrlPubMed
    16. ↵
      1. Kruijshaar ME,
      2. Lipman M,
      3. Essink-Bot M-L, et al
      . Health status of UK patients with active tuberculosis. Int J Tuberc Lung Dis 2010;14:296–302.
      OpenUrlPubMed
    17. ↵
      House of Commons Health Committee. Health Inequalities. House of Commons, London; 2009 Mar p. 28. Report No. Volume I.
    18. ↵
      1. Sheringham J,
      2. Baraitser P,
      3. Simms I, et al
      . Chlamydia screening in England: a qualitative study of the narrative behind the policy. BMC Public Health 2012;12:317. doi:10.1186/1471-2458-12-317
      OpenUrlCrossRefPubMed
    19. ↵
      1. Veen J
      . Microepidemics of tuberculosis: the stone-in-the-pond principle. Tuber Lung Dis 1992;73:73–6. doi:10.1016/0962-8479(92)90058-R
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. De Beer JL,
      2. van Ingen J,
      3. de Vries G, et al
      . Comparative study of IS6110 restriction fragment length polymorphism and variable-number tandem-repeat typing of Mycobacterium tuberculosis isolates in the Netherlands, based on a 5-year nationwide survey. J Clin Microbiol 2013;51:1193–8. doi:10.1128/JCM.03061-12
      OpenUrlAbstract/FREE Full Text
    21. ↵
      1. Bryant JM,
      2. Schürch AC,
      3. van Deutekom H, et al
      . Inferring patient to patient transmission of Mycobacterium tuberculosis from whole genome sequencing data. BMC Infect Dis 2013;13:110. doi:10.1186/1471-2334-13-110
      OpenUrlCrossRefPubMed
    22. ↵
      1. De Vries G,
      2. van Hest RAH,
      3. Richardus JH
      . Impact of mobile radiographic screening on tuberculosis among drug users and homeless persons. Am J Respir Crit Care Med 2007;176:201–7. doi:10.1164/rccm.200612-1877OC
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Ruddy MC,
      2. Davies AP,
      3. Yates MD, et al
      . Outbreak of isoniazid resistant tuberculosis in north London. Thorax 2004;59:279–85. doi:10.1136/thx.2003.010405
      OpenUrlAbstract/FREE Full Text
    24. ↵
      1. Malakmadze N,
      2. González IM,
      3. Oemig T, et al
      . Unsuspected recent transmission of tuberculosis among high-risk groups: implications of universal tuberculosis genotyping in its detection. Clin Infect Dis 2005;40:366–73. doi:10.1086/427112
      OpenUrlAbstract/FREE Full Text
    25. ↵
      1. Lambregts-van Weezenbeek CSB,
      2. Sebek MMGG,
      3. van Gerven PJHJ, et al
      . Tuberculosis contact investigation and DNA fingerprint surveillance in The Netherlands: 6 years’ experience with nation-wide cluster feedback and cluster monitoring. Int J Tuberc Lung Dis 2003;7(12 Suppl 3):S463–470.
      OpenUrlPubMedWeb of Science
    26. ↵
      1. Borgdorff MW,
      2. van Soolingen D
      . The re-emergence of tuberculosis: what have we learnt from molecular epidemiology? Clin Microbiol Infect 2013;19: 889–901. doi:10.1111/1469-0691.12253
      OpenUrlCrossRefPubMed
    27. ↵
      1. Walker TM,
      2. Ip CL,
      3. Harrell RH, et al
      . Whole-genome sequencing to delineate Mycobacterium tuberculosis outbreaks: a retrospective observational study. Lancet Infect Dis 2013;13:137–46. doi:10.1016/S1473-3099(12)70277-3
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Small PM,
      2. Shafer RW,
      3. Hopewell PC, et al
      . Exogenous reinfection with multidrug-resistant Mycobacterium tuberculosis in patients with advanced HIV infection. N Engl J Med 1993;328:1137–44. doi:10.1056/NEJM199304223281601
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Borgdorff MW,
      2. van den Hof S,
      3. Kremer K, et al
      . Progress towards tuberculosis elimination: secular trend, immigration and transmission. Eur Respir J 2010;36:339–47. doi:10.1183/09031936.00155409
      OpenUrlAbstract/FREE Full Text
    30. ↵
      1. De Beer JL,
      2. Kodmon C,
      3. van der Werf MJ, et al
      ., ECDC MDR-TB Molecular Surveillance Project Participants. Molecular surveillance of multi- and extensively drug-resistant tuberculosis transmission in the European Union from 2003 to 2011. Euro Surveill 2014;19:pii: 20742.
      OpenUrlCrossRef

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • First three authors and two final authors in order of contribution. All other authors in alphabetical order.

    • Correction notice This article has been corrected since it was published Online First. The provenance and peer review statement has been corrected.

    • Contributors The TB-STS Evaluation was initiated by IA and PS. PS chaired the Evaluation Group, which included EV, JL, MWB, TC, DC, JAI, ML, HM, TDM and IA. JM collected the data, conducted the analyses and wrote the findings. EV developed the mathematical model. JL conducted the cost-effectiveness analyses. For this manuscript, all authors contributed to the interpretation and writing of the findings. The corresponding author had full access to all the data in the study and has final responsibility for the decision to submit for publication.

    • Funding This study was funded by a Public Health England Research and Development Bid and a UCL IMPACT Studentship. The TB-STS Evaluation was supported by Public Health England's TB Strain-typing Project Board. JM is funded by a University College London and Public Health England Impact Studentship. IA is funded by an NIHR Senior Research Fellowship and the MRC. The sponsors played no role in the study design, data interpretation, data analysis, writing of the report or the decision to submit for publication.

    • Competing interests JM, IA, HM, ML and EV have been employed by Public Health England in the last five years. There are no other potential conflicts of interest.

    • Ethics approval The study was classified as a service evaluation by University College London Hospital Foundation Trust; therefore, specific ethical approval was not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement The manuscript is an honest, accurate and transparent account of the study being reported; no important aspects of the study have been omitted.

    Linked Articles