arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2404.13164

Least Squares Estimation For Hierarchical Data

Ryan Cumings-Menon·Pavel Zhuravlev
View original source ↗Read source document ↗Export metadata ↓

Abstract

The U.S. Census Bureau's 2020 Disclosure Avoidance System (DAS) bases its output on noisy measurements, which are population tabulations added to realizations of mean-zero random variables. These noisy measurements are observed for a set of hierarchical geographic levels, e.g., the U.S. as a whole, states, counties, census tracts, and census blocks. The Census Bureau released the noisy measurements generated in the DAS executions for the two primary 2020 Census data products, in part to allow data users to assess uncertainty in 2020 Census tabulations introduced by disclosure avoidance. This paper describes an algorithm that can leverage the hierarchical structure of the input data in order to compute very high dimensional least squares estimates in a computationally efficient manner. Afterward, we show that this algorithm's output is equal to the generalized least squares estimator, describe how to find the variance of linear functions of this estimator, and provide a numerical experiment in which we compute confidence intervals of tabulations based on this estimator. We also describe an accompanying Census Bureau experimental data product that applies this estimator to the publicly available noisy measurements to provide data users with the inputs required to derive confidence intervals for all tabulations that were included in the 2020 Redistricting Data File, for the U.S., state, county, and census tract geographic levels.

Explore related subjects

stat.COcs.CR

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Ryan Cumings-Menon, Pavel Zhuravlev. 2026-05-01. Least Squares Estimation For Hierarchical Data. https://arxiv.org/abs/2404.13164

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Meta-analysis with the glmmTMB R package

Two common formulations of meta-analytical models include the standard two-stage normal-normal models, which synthesise estimated effect sizes, and the one-stage generalised linear mixed model (GLMM), which directly model the underlying outcome data across studies. The general-purpose glmmTMB R package provides flexible response distributions and random-effect covariance structures through Template Model Builder (TMB). Its existing functionality can fit one-stage meta-analytic GLMM specifications. However, incorporating known sampling variances and covariances in the conventional two-stage inverse-variance formulation of meta-analysis was previously not easily accomplished in glmmTMB. Here, we introduce equalto, a new covariance structure in glmmTMB that allows users to supply a known sampling error variance-covariance matrix when fitting meta-analytic models. This enables explicit modelling of heteroscedasticity and dependence among sampling errors. Using simulations, we show that glmmTMB produces estimates identical to those from the corresponding metafor package functions for normal-normal models and similar estimates for GLMM specifications. We illustrate these models using published meta-analysis datasets in medicine, evolutionary ecology, and the social sciences. With the addition of the equalto covariance structure, glmmTMB now provides a unified and flexible framework for fitting two-stage normal-normal models and one-stage meta-analytic GLMMs, including multivariate specifications. These models can be fitted using the same glmmTMB() function, expanding the R toolkit available for evidence synthesis.

stat.CO↗
arXiv2026

Edgewise Envelopes Between Balanced Forman and Ollivier-Ricci Curvature

Evaluating Ollivier-Ricci (OR) curvature on large-scale graphs is computationally prohibitive due to the necessity of solving an optimal transport problem for every edge. We bypass this bottleneck by deriving explicit, two-sided, piecewise-affine transfer moduli between the transport-based OR curvature and the combinatorial Balanced Forman (BF) curvature. We establish deterministic bounds for $\mathfrak{c}_{\rm OR}(i,j)$ parameterized by 2-hop local graph combinatorics, reducing the edgewise evaluation complexity from an optimal transport linear program to a worst-case $\mathcal{O}\left(\max_{v \in V} \operatorname{deg}(v)^{2.5}\right)$ time, entirely eliminating the reliance on global solvers. Empirical scalability benchmarks confirm these theoretical guarantees, demonstrating that the proposed transfer moduli yield significant asymptotic and constant-factor speedups over the steep polynomial scaling of exact OR evaluation. Furthermore, the tightness of these bounds is validated via distributional analyses on canonical random graphs and empirical networks, with the derived analytical bands enclosing the empirical distributions independent of degree heterogeneity, geometry, or clustering, providing a scalable, computationally efficient framework for rigorous statistical network analysis.

stat.CO↗
arXiv2026

Markov Chain Monte Carlo with Diffusion Paths

Sampling from multimodal distributions is a longstanding challenge for classical local Markov chain Monte Carlo (MCMC) methods. A popular remedy is to introduce a sequence of intermediate distributions that interpolate between the target and a simpler reference. The classical choice, tempering, raises the density to a power, but distorts the relative weights of asymmetric modes and can lead to poor mixing. We instead propose interpolating along the diffusion path, the marginals of a noising diffusion process that carries the target toward a Gaussian. This path preserves the relative weights of the modes and enjoys favorable mixing properties, which we make precise through a spectral-gap analysis of the corresponding ideal transition kernel. Sampling along the path requires its intermediate scores, which can be estimated from the unnormalized target through variational approaches, yielding only an approximate sampler. To remove the resulting bias, we introduce the Metropolis-adjusted diffusion path (MAD-Path) sampler, which corrects the diffusion-path proposal in an augmented path space and leaves the target invariant regardless of the accuracy of the learned score or the discretization error. We further quantify how these two errors affect the acceptance probability, providing guidance for practical tuning. Experiments on a range of Bayesian posteriors show that MAD-Path improves global exploration and mode-weight estimation relative to tempering-based MCMC methods and unadjusted diffusion samplers.

stat.CO↗