arXiv Science⌕ Search

arXiv · 2610.39995

How Different are Different Differences-in-Differences? A Large-Scale Meta-Analysis

Abstract

I use AI-assisted replication to assess the empirical importance of negative weights in two-way fixed effects (TWFE) estimation, analyzing 1,430 staggered-adoption specifications from 116 economics papers. TWFE estimates are highly correlated with modern difference-in-differences estimates, with sign reversals in 12.0-14.9% of comparisons and greater precision for TWFE. Hierarchical models accounting for estimation noise imply estimand correlations of 91-97% and a 0.7-2.4% probability of a statistically significant TWFE estimate opposing the modern estimand's sign. These findings suggest negative weights may be a less serious problem in practice and illustrate how AI can support systematic, large-scale empirical assessments of new econometric methods.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Peter Hull. 2026-11-03. How Different are Different Differences-in-Differences? A Large-Scale Meta-Analysis. https://arxiv.science/abs/2610.39995

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Difference-in-Differences Estimators for Treatments Continuously Distributed at Every Period

When studying the effects of taxes, tariffs, or prices using panel data, the treatment is often continuously distributed in every period. We develop difference-in-differences (DID) estimators for such settings. We partition units into switchers, whose treatment changes between consecutive periods, and stayers, whose treatment remains constant. Under a parallel-trends assumption, we show that the slopes of switchers' potential outcomes with respect to the treatment are nonparametrically identified by DID comparisons between switchers and stayers sharing the same baseline treatment level. Conditioning on the baseline treatment is key, as it ensures that the underlying parallel-trends assumption accommodates time-varying treatment effects. We then study two weighted averages of these slopes, discuss their respective advantages, and propose for each a doubly robust, semiparametrically efficient, and $\sqrt{n}$-consistent estimator. Finally, we extend our framework to instrumental variables and illustrate it by estimating the effects of gasoline taxes on prices and fuel consumption.

econ.EM↗

Difference-in-differences for mediation analysis using double machine learning

We propose a difference-in-differences (DiD) framework with mediation for possibly multivalued discrete or continuous treatments and mediators, aimed at identifying the direct effect of the treatment on the outcome (net of effects operating through the mediator), the indirect effect via the mediator, and the joint effects of treatment and mediator, consistent with the framework of dynamic treatment effects. Identification relies on a conditional parallel trends assumption imposed on the mean potential outcome across treatment and mediator states, or (depending on the causal parameter) additionally on the mean potential outcomes and potential mediator distributions across treatment states. We propose ATET estimators for repeated cross sections and panel data within the double/debiased machine learning framework, which allows for data-driven control of covariates, and we establish their asymptotic normality under standard regularity conditions. We investigate the finite-sample performance of the proposed methods in a simulation study and illustrate our approach in an empirical application to the US National Longitudinal Survey of Youth, estimating the direct effect of health care coverage on general health as well as the indirect effect operating through routine checkups.

econ.EM↗

The Online LSTM Prediction Models and The Term Structure of Predictions

We examine and illustrate the properties of the Long Short Term Memory (LSTM) prediction models applied online to stationary inputs. We discuss the effects of: a) the network architecture that defines the admissible prediction functions, b) the objective function used to measure the prediction accuracy, and c) the Stochastic Gradient Descent (SGD) algorithm. We also study the prediction optimality, the term structure of predictions, the filtered memory dynamics. We also introduce a set of diagnostic tools for memory-based prediction models that focus on the estimated dynamics of the hidden states.

econ.EM↗