arXiv subjects

T. D. Stanley

Publications and source records attributed to T. D. Stanley.

4 recordsLinked to original sources

arXiv2026

Why the unrestricted weighted least squares should be routinely reported in medical meta-analyses

The unrestricted weighted least squares (UWLS) meta-analysis estimator of mean effect is an alternative to the conventional random-effects model (RE). It is a weighted least squares regression estimator that can be represented as a multiplicative random-effects model. UWLS has been shown to fit medical research better than RE as measured by AIC/BIC widely across Cochrane Database of Systematic Reviews (CDSR). The independence of UWLS's mean and heterogeneity estimators provide small-sample advantages that RE does not possess. Large small-sample biases and uncertainty in RE's heterogeneity variance estimates explain most of RE's relatively poor fit along with RE's boundary problem, where RE's heterogeneity variance is estimated to be zero. We prove that UWLS almost always has superior fit at RE's boundary with uncommon exceptions. UWLS has also been found to have generally superior statistical properties: bias, MSE, and coverage relative to RE across 1,665 simulation designs compiled from four published studies authored by different teams of researchers. A recent study in this journal replicated UWLS's superior goodness of fit widely across both the CDSR and a new set of simulations. Due to reporting and interpretation errors, this recent study calls for the continued use of RE as the default meta-analysis estimator with limited applications of UWLS. We address this recent study's concerns and show why UWLS should be routinely reported in medical meta-analyses.

stat.ME↗

arXiv2026

Do methods matter in the meta-analysis of partial correlation coefficients?

Recent studies have demonstrated that conventional meta-analyses of partial correlation coefficients (PCC) are biased. Several adjustments have been shown in simulations to reduce these small-sample biases to negligibility. While many meta-analyses of partial correlation coefficients are conducted each year across several disciplines, the practical importance of these issues remains unknown. To address this question and to offer advice for applications, we survey 172 economic meta-analyses of PCCs. We find that small-sample biases are negligible in practice. However, some publication selection biases remain. Although Fisher's z transformations have often been recommended, they reduce neither small-sample nor publication selection biases relative to conventional random effects. Both the unrestricted weighted least squares (UWLS) and the Hunter-Schmidt (HS) estimators produce smaller, arguably less biased, estimates of the mean PCC in these applications than either random effects with or without Fisher's z transformations. These findings offer practical guidance for any discipline that meta-analyzes partial correlations.

econ.EM↗

arXiv2026

Do decisions about outliers and influential effects matter? Evidence from 358 behavioral science meta-analyses

Meta-analysts routinely face estimates that look too large or extreme. Yet, how to handle them is left to the reviewer's judgment. The methods for detecting such estimates are well known. What is missing is an informed assessment of how much alternative handling choices might change a meta-analysis' conclusions. We fill this gap by analyzing the effects of four pre-registered handling treatments across 358 behavioral science meta-analyses with at least ten estimates. Each outlier handling treatment is estimated by two estimators (random effects and unrestricted weighted least squares), and compared to the 'do-nothing' baseline on three outcomes: the pooled effect, statistical significance, and whether the effect reaches the smallest effect size of interest (|d| >= 0.20). Our entire analysis and comparison pipelines were pre-registered. Alternative outlier handling treatments have little effect on the meta-analysis mean as the median absolute change in Cohen's d is at most 0.047 and often much less. Yet, at least one of these four treatments in combination with one of these estimators reverses the statistical significance of 11.5% of meta-analyses and the smallest-effect-of-interest assessment in 15.9%. Winsorizing has the least effect and DFBETAS the most. Categorical changes are found almost entirely among results already close to the decision boundary; strongly significant results essentially never change. These findings give applied meta-analysts, methods specialists, and reviewers a reference point for how much this under-reported choice matters and provide yet another reason for meta-analysts to publicly pre-specify their methods and handling treatments.

econ.EM↗

arXiv2023

Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics

Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.

stat.AP↗