Standardized mean differences cause funnel plot distortion in publication bias assessments
- University Medical Center Utrecht, Netherlands
- University of Edinburgh, United Kingdom
- University of Tasmania, Australia
- Radboud Institute for Health Sciences, Radboud University Medical Center, Netherlands
- The University of Edinburgh, United Kingdom
- Radboud University Medical Center, Netherlands
Abstract
Meta-analyses are increasingly used for synthesis of evidence, and often include an assessment of publication bias based on detection of asymmetry in funnel plots. We studied the influence of different normalisation approaches, sample size and intervention effects on funnel plot asymmetry, using empirical datasets and illustrative simulations. We found that funnel plots of the Standardized Mean Difference (SMD) plotted against the standard error (SE) are susceptible to distortion, leading to overestimation of the existence and extent of publication bias. Distortion was more severe when the primary studies had a small sample size and when an intervention effect was present. We show that using the Normalised Mean Difference (when possible), or plotting the SMD against a sample size-based precision estimate, are more reliable alternatives. We conclude that funnel plots using the SMD in combination with the SE are unsuitable for publication bias assessments and can lead to false-positive results.
Article and author information
Author details
Funding
National Institute of Environmental Health Sciences (National Toxicology Program research funding)
- Kimberley E Wever
Netherlands Cardiovascular Research Initiative (CVON-HUSTCARE)
- Steven AJ Chamuleau
National Centre for the Replacement, Refinement and Reduction of Animals in Research (Infrastructure Award)
- Emily S Sena
- Malcolm R MacLeod
Alexander Suerman program (PhD student Scholarship)
- Peter-Paul Zwetsloot
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Reviewing Editor
- M Dawn Teare, University of Sheffield, United Kingdom
Version history
- Received: December 14, 2016
- Accepted: August 21, 2017
- Accepted Manuscript published: September 8, 2017 (version 1)
- Version of Record published: September 29, 2017 (version 2)
Copyright
© 2017, Zwetsloot et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 5,859
- views
-
- 430
- downloads
-
- 128
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Standardized mean differences cause funnel plot distortion in publication bias assessments
eLife 6:e24260.
https://doi.org/10.7554/eLife.24260
Further reading
-
- Epidemiology and Global Health
We discuss 12 misperceptions, misstatements, or mistakes concerning the use of covariates in observational or nonrandomized research. Additionally, we offer advice to help investigators, editors, reviewers, and readers make more informed decisions about conducting and interpreting research where the influence of covariates may be at issue. We primarily address misperceptions in the context of statistical management of the covariates through various forms of modeling, although we also emphasize design and model or variable selection. Other approaches to addressing the effects of covariates, including matching, have logical extensions from what we discuss here but are not dwelled upon heavily. The misperceptions, misstatements, or mistakes we discuss include accurate representation of covariates, effects of measurement error, overreliance on covariate categorization, underestimation of power loss when controlling for covariates, misinterpretation of significance in statistical models, and misconceptions about confounding variables, selecting on a collider, and p value interpretations in covariate-inclusive analyses. This condensed overview serves to correct common errors and improve research quality in general and in nutrition research specifically.
-
- Ecology
- Epidemiology and Global Health
Zoonotic disease dynamics in wildlife hosts are rarely quantified at macroecological scales due to the lack of systematic surveys. Non-human primates (NHPs) host Plasmodium knowlesi, a zoonotic malaria of public health concern and the main barrier to malaria elimination in Southeast Asia. Understanding of regional P. knowlesi infection dynamics in wildlife is limited. Here, we systematically assemble reports of NHP P. knowlesi and investigate geographic determinants of prevalence in reservoir species. Meta-analysis of 6322 NHPs from 148 sites reveals that prevalence is heterogeneous across Southeast Asia, with low overall prevalence and high estimates for Malaysian Borneo. We find that regions exhibiting higher prevalence in NHPs overlap with human infection hotspots. In wildlife and humans, parasite transmission is linked to land conversion and fragmentation. By assembling remote sensing data and fitting statistical models to prevalence at multiple spatial scales, we identify novel relationships between P. knowlesi in NHPs and forest fragmentation. This suggests that higher prevalence may be contingent on habitat complexity, which would begin to explain observed geographic variation in parasite burden. These findings address critical gaps in understanding regional P. knowlesi epidemiology and indicate that prevalence in simian reservoirs may be a key spatial driver of human spillover risk.
