arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.10435

On the Benefit of Blocking for Online Experiments with Skewed Data

Abbas Zaidi·Matthew Reimherr·Rina Friedberg·Fabian Martinez·Richard Mudd
View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper advocates for using blocked (stratified) assignment in online A/B tests to handle highly skewed population data. While blocking yields only modest precision gains (5-10%) due to the limits of discretization, the authors demonstrate it offers two crucial structural benefits over post-hoc statistical adjustments. First, fixed-weight blocking correctly targets the true average treatment effect, avoiding the severe bias introduced by efficiency-weighted alternatives. Second, blocking localizes extreme outliers, enabling targeted within-block winsorization that effectively controls noise without destroying the tail-end treatment effect.

Explore related subjects

stat.AP

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Abbas Zaidi, Matthew Reimherr, Rina Friedberg, Fabian Martinez, Richard Mudd. 2026-10-07. On the Benefit of Blocking for Online Experiments with Skewed Data. https://arxiv.org/abs/2610.10435

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Spatial Dependencies in Item Response Theory: Gaussian Process Priors for Ecological and Cognitive Measurement

Measurement validity in Item Response Theory depends on appropriately modeling dependencies between items when these reflect meaningful theoretical structures rather than random measurement error. In ecological assessment, citizen scientists identifying species across geographic regions exhibit systematic spatial patterns in task difficulty due to environmental factors. Similarly, in Author Recognition Tests, literary knowledge organizes by genre, where familiarity with science fiction authors systematically predicts recognition of other science fiction authors. Current spatial Item Response Theory methods, represented by the 1PLUS, 2PLUS, and 3PLUS model family, address these dependencies but remain limited by (1) binary response restrictions, and (2) conditional autoregressive priors that impose rigid local correlation assumptions, preventing effective modeling of complex spatial relationships. Our proposed method, Spatial Gaussian Process Item Response Theory (SGP-IRT), addresses these limitations by replacing conditional autoregressive priors with flexible Gaussian process priors that adapt to complex dependency structures while maintaining principled uncertainty quantification. SGP-IRT accommodates polytomous responses and models spatial dependencies in both geographic and abstract cognitive spaces, where items cluster by theoretical constructs rather than physical proximity. Simulation studies demonstrate improved parameter recovery, particularly for item difficulty estimation. Empirical applications show enhanced recovery of meaningful difficulty surfaces and improved measurement precision across psychological, educational, and ecological research applications.

stat.AP↗
arXiv2026

Day-Ahead Electricity Price Forecasting Using a Multivariate Group Lasso Method

Electricity price signals in modern power systems exhibit complex dependence structures that render forecasting inherently challenging. Our analysis of real-world electricity pricing signals reveals complex temporal group effects, whereby the influence of explanatory variables on electricity prices persists across consecutive blocks of time due to underlying economic, system, and operational drivers. In response, we propose a multivariate statistical method based on a Group Lasso formulation to jointly forecast the vector of day-ahead electricity prices (h = 1, ..., 24), by leveraging multi-feature temporal group effects. Our approach is evaluated on two full years of electricity prices from the California Independent System Operator (CAISO), demonstrating considerable improvements in point and probabilistic forecast metrics compared to a wide array of statistical and deep learning methods. Empirical analyses confirm the effectiveness of the proposed approach in modeling realistic group effects, maintaining both interpretability and low computational complexity. When retrospectively evaluated on test data from a recent international electricity price forecasting challenge, the proposed method ranked in second place, despite having access to significantly less information than competing approaches. Finally, the proposed method is independently validated against two operational electricity price forecasting systems in CAISO, demonstrating competitive predictive performance and practical relevance.

stat.AP↗
arXiv2026

Testing Additivity of Lead and Benzo[a]pyrene-induced Neurotoxicity in Caenorhabditis elegans Assays

Exposure to environmental contaminants is a recognized cause of neurotoxicity, contributing to the onset of a broad range of neurological conditions. In realistic settings, such exposure involves com- plex mixtures, and the combined effect of their components may differ from what their individual effects would predict. Characterizing such interactions and testing them against a principled notion of additivity is central to assessing the neurotoxicological risk. We take up these questions for two widespread and independently neurotoxic pollutants, lead (Pb) and benzo[a]pyrene (BaP), through a novel C. elegans assay in which nematodes were subjected to single and joint exposures across a range of doses. Morphological damage is quantified on an ordinal scale at the level of individual dopaminergic neurons. To analyze these data, we model the full distribution of the ordinal response as a convex mixture between an unexposed and a maximally affected profile. The weight of this mixture varies with chemical doses, modeled flexibly via monotone splines and, for the joint effect, in a radial coordinate system. Additivity is assessed via a likelihood ratio test against established null models, and is calibrated via parametric bootstrap. Applied to the C. elegans assay, our analysis reveals a localized, asymmetric synergy between Pb and BaP, concentrated where moderate BaP meets high Pb exposure.

stat.AP↗