arXiv Science⌕ Search

arXiv · 2611.12647

Near-Purity Screening and Dominant-Atom Estimation in Softmax Mixtures and Their Ensembles

Abstract

We study near-purity screening and dominant-atom estimation in softmax mixtures, whose components share a common set of category features. A mixture is called $\varepsilon$-pure if one of its mixing weights is at least $1-\varepsilon$. We introduce the Trace of Latent Covariance Criterion ($\mathrm{TLC}^{2}$), a direct estimator of the trace of the covariance matrix of the latent mixing measure, constructed without estimating its atoms or weights, or knowing the number of components. Unlike existing moment-based approaches, our estimator is constructed for the softmax family and does not require knowledge or estimation of the feature distribution. We establish finite-sample guarantees for $\mathrm{TLC}^{2}$, showing that it achieves a parametric convergence rate up to an additive approximation error of order $p^{-1/2}$, without requiring minimum separation between mixture atoms. We also study estimation of the dominant atom in near-pure mixtures by fitting a single softmax component. We prove that the resulting maximum likelihood estimator has an error bound consisting of the usual parametric term and a bias term governed by the weighted distances between the dominant and remaining atoms, revealing adaptation to the geometry of the mixture. Finally, we extend near-purity screening and dominant-atom estimation to ensembles of softmax mixtures with shared atoms and mixture-specific weights. We apply $\mathrm{TLC}^{2}$ separately to each ensemble member to screen for near-purity without knowing the number of components. Under additional assumptions, clustering the resulting dominant-atom estimators yields finite-sample guarantees for estimating the shared atoms. An application to the Million Song Dataset illustrates how the estimated softmax directions reveal interpretable structure in the data.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Xin Bing, Florentina Bunea, Marten Wegkamp, Weicheng Wu. 2026-11-09. Near-Purity Screening and Dominant-Atom Estimation in Softmax Mixtures and Their Ensembles. https://arxiv.science/abs/2611.12647

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning

Across a wide array of disciplines, many researchers use machine learning (ML) algorithms to identify a subgroup of individuals who are likely to benefit from a treatment the most (``exceptional responders'') or those who are harmed by it. A common approach to this subgroup identification problem consists of two steps. First, researchers estimate the conditional average treatment effect (CATE) using an ML algorithm. Next, they use the estimated CATE to select those individuals who are predicted to be most affected by the treatment, either positively or negatively. Unfortunately, CATE estimates are often biased and noisy. In addition, utilizing the same data to both identify a subgroup and estimate its group average treatment effect results in a multiple testing problem. To address these challenges, we develop uniform confidence bands for estimation of the group average treatment effect sorted by generic ML algorithm (GATES). Using these uniform confidence bands, researchers can identify, with a statistical guarantee, a subgroup whose GATES exceeds a certain effect size, regardless of how this effect size is chosen. The proposed methodology uses random treatment assignment, random sampling of units, and continuity and moment conditions, without requiring consistency or monotonicity of the ML score. We derive a Wiener representation of the estimation error, with corrections for the empirical subgroup threshold and the fixed treatment sample size. Estimating this representation gives asymptotically exact uniform coverage using Gaussian draws, without refitting the ML model. We also develop uniform bands for mean-adjusted treatment effects and other evaluation metrics. A simulation design and an application to a clinical trial of late-stage prostate cancer illustrate the subgroup identification problem.

stat.ME↗

Collaborative Optimal Design of Controlled Experiments with Subject Covariates

Modern experimental design problems often encounter multiple controlled experiments with subjects participating in more than one experiment. It is of interest to consider the optimal design of allocating subjects to treatment or control for these experiments especially when subject covariate information is available. In this work, we propose an optimal allocation method to enable accurate estimates of treatment effects for all experiments. Deriving the precision matrix of the treatment effects and using D-optimality as our allocation criterion, we demonstrate the advantage of collaboratively designing and analyzing multiple experiments over traditional independent design and analysis, and propose efficient algorithms to design construction of the D-optimality problem for collaborative experiments. The performance of the proposed methods is examined by simulation studies and a case study involving online education experimentation.

stat.ME↗

Parameter-Specific Bias Diagnostics in Random-Effects Panel Data Models

The Hausman specification test assesses the random-effects specification by comparing random-effects and fixed-effects estimators. This note uses the diagnostic of Karl and Zimmerman (2021) to give a coefficient-specific sensitivity summary from one fitted mixed model. The diagnostic measures alignment between fitted random effects and the weights used to estimate a coefficient or contrast. Its value need not equal finite-sample bias: fitted effects are shrunken, and response-dependent covariance estimates introduce an additional weighted-error term. We explain this distinction, describe simulations that refit the covariance parameters and retain the latent-effect and weighted-error contributions, and illustrate the calculations with gasoline-demand data and a multiple-membership value-added model. Connecting the diagnostic to established generalized least squares sensitivity analysis, including Vasnev's (2010) balanced-panel results, we identify it as the negative derivative of a coefficient or contrast with respect to the log scale of the random-effects covariance. With fitted covariance shapes held fixed, integrating the diagnostic over this log scale, upward from the fitted model, gives the random-effects estimate minus the fixed-effects estimate for a contrast estimable under both models. We also discuss conditioning on stochastic regressors and the additional assumptions needed for calibrated inference.

stat.ME↗