Review Article
Use of methodological search filters to identify diagnostic accuracy studies can lead to the omission of relevant studies

https://doi.org/10.1016/j.jclinepi.2005.07.014Get rights and content

Abstract

Objective

To determine the usefulness of methodological filters in search strategies for diagnostic studies in systematic reviews.

Study Design and Setting

We made an inventory of existing methodological search filters for diagnostic accuracy studies and applied them in PubMed to a reference set derived from 27 published systematic reviews in a broad range of clinical fields. Outcome measures were the fraction of not identified relevant studies and the reduction in the number of studies to read.

Results

We tested 12 search filters. Of the studies included in the systematic reviews, 2%–28% did not pass the sensitive search filters, 4%–24% did not pass the accurate filters, and 39%–42% did not pass the specific filters. Decrease in number-needed-to-read when a search filter was used in a search strategy for a diagnostic systematic review varied from 0% to 77%.

Conclusion

The use of methodological filters to identify diagnostic accuracy studies can lead to omission of a considerable number of relevant studies that would otherwise be included. When preparing a systematic review, it may be preferable to avoid using methodological filters.

Introduction

Systematic reviews, regarded as the cornerstones of evidence-based medicine, are intended to identify and evaluate all available evidence about a specific topic. A systematic and comprehensive search for relevant primary studies is one of the essential steps in conducting a systematic review, and one of the factors that distinguishes a systematic review from a traditional narrative review.

To identify diagnostic accuracy studies in an electronic database, such as Medline, several search strategies can be used. Many of these strategies rely on free or text words and MeSH headings directed to disease indicators in combination with search terms for the diagnostic test. To further limit the search results, a methodological filter can be used, consisting of text words and MeSH headings directed to general indicators of diagnostic studies. In contrast to their use with intervention studies, however, these general indicators are not widely and systematically used as keywords for diagnostic studies, and indexing of original diagnostic accuracy studies is not flawless [1], [2], [3]. Diagnostic studies show more variability in study design than intervention studies. It is therefore not unlikely that a considerable number of relevant studies will be missed when those filters are used in diagnostic reviews, and the reduction in number of studies to consider for inclusion is far from impressive.

A number of these methodological search filters for diagnostic accuracy studies have been validated. They are known to differ in sensitivity (percentage correctly identified studies) and specificity (percentage correctly nonidentified studies). Our objective was to assess the usefulness of these search filters by applying them to a reference set derived from published systematic reviews in a broad range of clinical fields. First, the fraction of relevant studies that did not pass each filter was calculated. Then we determined whether the diagnostic search filters focus the search strategy enough to be practical.

Section snippets

Methods

To identify articles reporting on the development of diagnostic search filters, we performed a computerized search using the databases Medline, EMBASE and the Cochrane Methodology Register of the Cochrane Library, with an end date of January 2004. The search terms used in Medline, interface PubMed, were: “Medline[MeSH] OR “Information Storage and Retrieval/methods”[MeSH]) AND diagnosis.” The search terms for EMBASE, interface OVID, were ((search adj strategy).mp. or (search adj strategies).mp.)

Results

Our search strategy for diagnostic search filters revealed a total number of 781 articles from three databases: Medline 605, EMBASE 95, and Cochrane Methodology Register 81. Eight articles were included [4], [5], [6], [7], [8], [9], [10], [11]. These articles described a total of 28 validated diagnostic search filters. Two studies described one accurate filter each [7], [8], and these filters were both selected as accurate filters. One study did not report sensitivity or specificity, nor

Discussion

Diagnostic reviews aim to identify and evaluate all available evidence about a specific index test or a comparison of tests. If the yield of the initial search based on index test and target condition is too large, a diagnostic search filter could be helpful in reducing this number. We compared the performance of 12 validated diagnostic search filters by applying them to a set of articles, selected from the reference lists of 27 published diagnostic systematic reviews. All filters studied may

References (42)

  • S.S. Sonnad et al.

    Accuracy of MR imaging for staging prostate cancer: a meta-analysis to examine the effect of technologic change

    Acad Radiol

    (2001)
  • A.M. Fielding et al.

    Using Medline to achieve an evidence-based approach to diagnostic clinical biochemistry

    Ann Clin Biochem

    (2002)
  • K. Dickersin et al.

    Identifying relevant studies for systematic reviews

    BMJ

    (1994)
  • N.L. Wilczynski et al.

    Reasons for the loss of sensitivity and specificity of methodologic MeSH terms and textwords in Medline

    Proc Annu Symp Comput Appl Med Care

    (1995)
  • L.M. Bachmann et al.

    Identifying diagnostic studies in Medline: reducing the number needed to read

    J Am Med Inform Assoc

    (2002)
  • R.B. Haynes et al.

    Developing optimal search strategies for detecting clinically sound studies in Medline

    J Am Med Inform Assoc

    (1994)
  • T. van der Weijden et al.

    Identifying relevant diagnostic studies in Medline: the diagnostic value of the erythrocyte sedimentation rate (ESR) and dipstick as an example

    Fam Pract

    (1997)
  • W.L. Devillé et al.

    [Systematic reviews in practice: X. Searching, selecting and the methodological assessment of diagnostic evaluation research]

    Ned Tijdschr Geneeskd

    (2002)
  • W.L. Devillé et al.

    Conducting systematic reviews of diagnostic studies: didactic guidelines

    BMC Med Res Methodol

    (2002)
  • S. Vincent et al.

    Clinical evidence diagnosis: developing a sensitive search strategy to retrieve diagnostic studies on deep vein thrombosis: a pragmatic approach

    Health Info Libr J

    (2003)
  • R.B. Haynes et al.

    Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey

    BMJ

    (2004)
  • Cited by (116)

    • Advances in the diagnosis and medical management of cushing’s syndrome

      2019, Advances in Treatment and Management in Surgical Endocrinology
    View all citing articles on Scopus
    View full text