Guidelines for Application of Metaanalysis in Environmental Epidemiology

doi:10.1006/rtph.1995.1084

Regulatory Toxicology and Pharmacology

Volume 22, Issue 2, October 1995, Pages 189-197

https://doi.org/10.1006/rtph.1995.1084 Get rights and content

Abstract

The use of meta-analysis in environmental epidemiology can enhance the value of epidemiologic data in debates about environmental health risks. Meta-analysis may be particularly useful to formally examine sources of heterogeneity, to clarify the relationship between environmental exposures and health effects, and to generate information beyond that provided by individual studies or a narrative review. However, meta-analysis may not be useful when the relationship between exposure and disease is obvious, when there are only a few studies of the key health outcomes, or when there is substantial confounding or other biases which cannot be adjusted for in the analysis. Recent increases in the use of meta-analysis in environmental epidemiology have highlighted the need for guidelines for the application of the technique. Guidelines, in the form of desirable and undesirable attributes, are presented in this paper for various components of a metaanalysis including study identification and selection; data extraction and analysis; and interpretation, presentation, and communication of results, Also discussed are the appropriateness of the use of meta-analysis in environmental health studies and when metaanalysis should or should not be used.

References (0)

Cited by (91)

Prenatal exposure to ambient air pollutants and congenital heart defects: An umbrella review
2023, Environment International
Prenatal exposure to ambient air pollutants has been linked to congenital heart defects (CHD), but findings of existing systematic reviews have been mixed.
To assess the epidemiological evidence on associations between prenatal exposure to ambient air pollutants and CHD subtypes, based on a systematic overview of reviews (“umbrella review”).
We conducted a systematic search for reviews assessing associations between prenatal exposure to criteria air pollutants and CHD. The risk of bias was evaluated using the Risk of Bias in Systematic Reviews (ROBIS) tool. The certainty of the systematic review findings was graded using the Navigation Guide methodology.
We identified eleven systematic reviews, including eight with meta-analyses, assessing in total 35 primary studies of prenatal exposure to criteria air pollutants and various CHD subtypes. The certainty of the findings of four meta-analyses indicating an increased risk for coarctation of the aorta associated with nitrogen dioxide exposure was rated as moderate. The certainty of findings indicating positive, inverse, or null associations for other pollutant-subtype combinations was rated as very low to low, based on low precision and high statistical heterogeneity of summary odds ratios (SOR), substantial inconsistencies between review findings, and methodological limitations of the systematic reviews.
The inconsistent findings and high statistical heterogeneity of many SOR of the included systematic reviews may partly be traced to differences in methodological approaches, and the risk of bias across included reviews (e.g., inclusion criteria, systematic search strategies, synthesis methods) and primary studies (e.g., exposure assessment, diagnostic criteria). Adherence to appropriate systematic review guidelines for environmental health research, as well as rigorous evaluation of risk of bias in primary studies, are essential for future risk assessments and policy-making. Still, our findings suggest that prenatal exposure to ambient air pollutants may increase risks for at least some CHD subtypes.
Application of artificial intelligence in clinical diagnosis and treatment: an overview of systematic reviews
2022, Intelligent Medicine
Citation Excerpt :
There is a longstanding consensus that publication bias affects the results of meta-analysis [80–81]. Inclusion of gray literature in the process of literature searching and determining the inclusion criteria can reduce the publication bias in meta-analyses to a certain extent [82–83]. The assessment of publication bias can also help evaluate how credible the evidence is [81].
This study aimed to summarize the characteristics and methodological quality of systematic reviews on the application of artificial intelligence (AI) in clinical diagnosis and treatment.
We systematically searched seven English- and Chinese-language literature databases to identify systematic reviews on the application of AI, deep learning, or machine learning in the diagnosis and treatment of any disease published in 2020. We evaluated the methodological quality of the included systematic reviews using “A Measurement tool for the assessment of multiple systematic reviews” (AMSTAR). We also conducted meta-analyses on the diagnostic accuracy of AI on selected disease categories with a large number of included studies and low clinical heterogeneity.
A total of 40 systematic reviews reporting 1,083 original studies were included, covering 31 diseases from 11 groups of diseases. Eleven systematic reviews were related to neoplasms and nine were systematic reviews related to diseases of the digestive system. We selected digestive system diseases for the meta-analysis. The pooled sensitivities (with 95% confidence interval (CI)) of AI to assist the diagnosis of helicobacter pylori, gastrointestinal ulcers, hemorrhage, esophageal tumors, gastric tumors, and intestinal tumors (with 95% CI) were 0.91 (0.83–0.95), 0.99 (0.76–1.00), 0.95 (0.83–0.99), 0.90 (0.85–0.93), 0.90 (0.82–0.95), and 0.93 (0.88–0.96), respectively, and the pooled specificities were 0.82 (0.77–0.87), 0.97 (0.86–1.00), 1.00 (0.99–1.00), 0.80 (0.71–0.87), 0.93 (0.87–0.97), and 0.89 (0.85–0.92), respectively. The AMSTAR items “the list of included studies” (n = 39, 97.5%) and “the characteristics of the included studies” (n = 39, 97.5%) had the highest compliance among the reviews; the compliance was relatively low to the items “the consideration of publication status” (n = 1, 2.5%), “the consideration of scientific quality” (n = 19, 47.5%), “data synthesis methods” (n = 18, 45.0%), and “ the evaluation of publication bias” (n = 13, 32.5%).
The main subjects of systematic reviews on AI applications in clinical diagnosis and treatment published in 2020 were diseases of the digestive system and neoplasms. The methodological quality of the systematic reviews on AI needs to be improved, paying particular attention to publication bias and the rigorous evaluation of the quality of the included studies.
Extreme heat and occupational injuries in different climate zones: A systematic review and meta-analysis of epidemiological evidence
2021, Environment International
The link between heat exposure and adverse health outcomes in workers is well documented and a growing body of epidemiological evidence from various countries suggests that extreme heat may also contribute to increased risk of occupational injuries (OI). Previously, there have been no comparative reviews assessing the risk of OI due to extreme heat within a wide range of global climate zones. The present review therefore aims to summarise the existing epidemiological evidence on the impact of extreme heat (hot temperatures and heatwaves (HW)) on OI in different climate zones and to assess the individual risk factors associated with workers and workplace that contribute to heat-associated OI risks.
A systematic review of published peer-reviewed articles that assessed the effects of extreme heat on OI among non-military workers was undertaken using three databases (PubMed, Embase and Scopus) without temporal or geographical limits from database inception until July 2020. Extreme heat exposure was assessed in terms of hot temperatures and HW periods. For hot temperatures, the effect estimates were converted to relative risks (RR) associated with 1 °C increase in temperature above reference values, while for HW, effect estimates were RR comparing heatwave with non-heatwave periods. The patterns of heat associated OI risk were investigated in different climate zones (according to Köppen Geiger classification) based on the study locations and were estimated using random-effects meta-analysis models.
Subgroup analyses according to workers’ characteristics (e.g. gender, age group, experience), nature of work (e.g. physical demands, location of work i.e. indoor/outdoor) and workplace characteristics (e.g. industries, business size) were also conducted.
A total of 24 studies published between 2005 and 2020 were included in the review. Among these, 22 studies met the eligibility criteria, representing almost 22 million OI across six countries (Australia, Canada, China, Italy, Spain, and USA) and were included in the meta-analysis. The pooled results suggested that the overall risk of OI increased by 1% (RR 1.010, 95% CI: 1.009–1.011) for 1 °C increase in temperature above reference values and 17.4% (RR 1.174, 95% CI: 1.057–1.291) during HW.
Among different climate zones, the highest risk of OI during hot temperatures was identified in Humid Subtropical Climates (RR 1.017, 95% CI: 1.014–1.020) followed by Oceanic (RR 1.010, 95% CI: 1.008–1.012) and Hot Mediterranean Climates (RR 1.009, 95% CI: 1.008–1.011). Similarly, Oceanic (RR 1.218, 95% CI: 1.093–1.343) and Humid Subtropical Climates (RR 1.213, 95% CI: 0.995–1.431) had the highest risk of OI during HW periods. No studies assessing the risk of OI in Tropical regions were found. The effects of hot temperatures on the risk of OI were acute with a lag effect of 1–2 days in all climate zones. Young workers (age < 35 years), male workers and workers in agriculture, forestry or fishing, construction and manufacturing industries were at high risk of OI during hot temperatures. Further young workers (age < 35 years), male workers and those working in electricity, gas and water and manufacturing industries were found to be at high risk of OI during HW.
This review strengthens the evidence on the risk of heat-associated OI in different climate zones. The risk of OI associated with extreme heat is not evenly distributed and is dependent on underlying climatic conditions, workers’ attributes, the nature of work and workplace characteristics. The differences in the risk of OI across different climate zones and worker subgroups warrant further investigation along with the development of climate and work-specific intervention strategies.
Statistics in endocrinology: Meta-analysis advantages and pitfalls
2018, Encyclopedia of Endocrine Diseases
A meta-analysis, technically a weighted average of the effect estimates of different studies, is generally assumed to provide high-level evidence. However, the validity of such an analysis ultimately depends on the quality and validity of included studies. The assessment of risk of bias of individual studies is thus crucial for every meta-analysis. Especially in the context of observational data, a thorough exploration of between study heterogeneity is probably more important than the mere production of a pooled estimate.
Ambient air pollution epidemiology systematic review and meta-analysis: A review of reporting and methods practice
2016, Environment International
Citation Excerpt :
Extracted data for each SRMA included: population characteristics, nature of AAP exposure, health outcomes, study designs used by underlying studies, and summary effect measures and confidence bounds, as well as responses to a questionnaire related to SRMA methods, reporting and strength of evidence evaluation. The questionnaire included 22 items we consolidated from several sources of “good-practice” guidance for SRMAs, including the 27-item PRISMA checklist for SRMAs (Moher et al., 2009), the 35-point MOOSE consensus guidelines for SRMAs in observational epidemiology, the Blair et al. 1995 recommendations for EH SRMAs, as well as more recent emerging SRMA guidance for EH from the NTP (Rooney et al., 2014) and Navigation Guide (Woodruff and Sutton, 2014). In selecting the 22 items we aimed for a simple questionnaire that would incorporate the core recommendations in four domains: (i) SRMA article reporting, including implications of research; (ii) systematic review search, selection and extraction methods; (iii) meta-analytic statistical pooling methods and approaches to examining heterogeneity, study quality and risk of bias; and (iv) methods for evaluating the strength of evidence.
Systematic review and meta-analysis (SRMA) are increasingly employed in environmental health (EH) epidemiology and, provided methods and reporting are sound, contribute to translating science evidence to policy. Ambient air pollution (AAP) is both among the leading environmental causes of mortality and morbidity worldwide, and of growing policy relevance due to health co-benefits associated with greenhouse gas emissions reductions.
We reviewed the published AAP SRMA literature (2009 to mid-2015), and evaluated the consistency of methods, reporting and evidence evaluation using a 22-point questionnaire developed from available best-practice consensus guidelines and emerging recommendations for EH. Our goal was to contribute to enhancing the utility of AAP SRMAs to EH policy.
We identified 43 studies that used both SR and MA techniques to examine associations between the AAPs PM_2.5, PM_10, NO₂, SO₂, CO and O₃, and various health outcomes. On average AAP SRMAs partially or thoroughly addressed 16 of 22 questions (range 10–21), and thoroughly addressed 13 of 22 (range 5–19). We found evidence of an improving trend over the period. However, we observed some weaknesses, particularly infrequent formal reviews of underlying study quality and risk-of-bias that correlated with lower frequency of thorough evaluation for key study quality parameters. Several other areas for enhanced reporting are highlighted.
The AAP SRMA literature, in particular more recent studies, indicate broad concordance with current and emerging best practice guidance. Development of an EH-specific SRMA consensus statement including a risk-of-bias evaluation tool, would be a contribution to enhanced reliability and robustness as well as policy utility.
Toward better research practice-Shortcomings decreasing the significance of epidemiological studies in the toxicological field
2014, NeuroToxicology
Citation Excerpt :
The increasing importance of these reviews also in the toxicological field may be concluded from the attempt of the National Toxicology Program to explore systematic-review methodology as a means to enhance transparency and increase efficiency in summarizing and synthesizing findings (Birnbaum et al., 2013). The usefulness of meta-analyses for hazard identification and estimates of dose–response curves was described already about 20 years ago (Blair et al., 1995). Since then meta-analyses have undergone a thorough and continuous quality assessment taking into account the most recent outcomes from research.
Neurobehavioral studies do not always gain the impact they should have, neither in the scientific nor in the regulatory field of neurotoxicology. Among others, shortcomings and inconsistencies across epidemiological studies may contribute to this situation. Examples were compiled to increase awareness of obstacles for conclusions. Meta-analyses were exploited since they sometimes allow the detection of deficits that are not obvious from individual studies.
Exposure assessment, performance measures, and confounding were scrutinized among 98 primary studies included in meta-analyses on mercury, solvents, manganese and pesticides.
Inconsistent and hardly comparable markers of exposure were found; figures, units or sampling periods were not always provided. The contribution of test materials to differences in test outcomes across studies could sometimes not be evaluated due to the insufficient description of the employed tests. Hypotheses for the selection of performance variables often remained undisclosed. Matching procedures prevailed with respect to the confounder age; the comparability of groups with respect to intelligence and gender remained more elusive. 8% and 16% of the studies did not even mention confounding from intelligence and gender, respectively. Only one third of the studies provided adjusted means for group comparisons; the proportion was slightly larger for studies published 2000–2010. While 50% of the studies considered confounders for their dose–response assessment, only 29% reported results for the total of test variables.
The outlined deficits impede, among others, the assessment of exposure–effect relationships and confounding across studies; thereby they limit the use of the studies for toxicological risk assessment and future prevention. Some shortcomings also impede a deeper insight into the mechanisms of toxicity: tests like the Digit Symbol show that something is affected, but not what is affected. Thorough description of measures employed is among the first consequences from the data. The consideration of mechanistic insights from research on animals and neurobiology may further help to increase the significance of epidemiological studies.

View all citing articles on Scopus

View full text

Regular ArticleGuidelines for Application of Metaanalysis in Environmental Epidemiology

Abstract

Regular Article
Guidelines for Application of Metaanalysis in Environmental Epidemiology