arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.28025

Optimal aggregation in multiclass classification

Felix Abramovich·Or Davidovitch-Servadio
View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper studies the problem of aggregation in multiclass classification, generalizing existing results for binary classification. Given a finite dictionary of score functions F={f_1,...,f_M} and a training sample, the goal is to construct a data-driven aggregate whose risk is as close as possible to that of the best aggregate (oracle) in a prescribed subset of span(F) with respect to a given loss. In particular, we investigate model selection and convex aggregation under the 0/1 and multinomial logistic losses. For the 0/1 loss, we show that the optimal aggregation rates remarkably do not depend on the number of classes K, and the corresponding empirical risk minimization (ERM) aggregates are rate-optimal for model selection and nearly-optimal for convex aggregation (up to an additional sqrt(ln(K)) factor). The situation under the multinomial logistic loss is more subtle. Although the optimal aggregation rates remain independent of K, restricting aggregation to proper aggregates in conv(F) leads to a deterioration of the achievable rates with K in certain regimes. This reveals a fundamental limitation of proper aggregation in multiclass classification. To overcome it, we propose improper probability-mixture aggregates that average the induced class probabilities rather than the original score functions and map the result back to score-function space. These aggregates achieve optimal or nearly optimal convergence rates independent of K. Finally, we introduce a different aggregation framework in which both the aggregation class and the oracle are defined directly in probability space.

Explore related subjects

math.STstat.MEstat.MLstat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Felix Abramovich, Or Davidovitch-Servadio. 2026-10-20. Optimal aggregation in multiclass classification. https://arxiv.org/abs/2610.28025

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Detection Is Harder Than Estimation in Certain Regimes: Inference for Moment and Cumulant Tensors

We study estimation and detection of high-order cumulant tensors from $n$ i.i.d. observations of a $p$-dimensional random vector, with performance measured in the tensor spectral norm. Under sub-Gaussianity, we show that the minimax rate for estimating the order-$d$ cumulant tensor is $\sqrt{p/n}\wedge 1$. In contrast to covariance estimation, the natural plug-in estimator obtained by replacing population moments in the moment--cumulant formula with sample moments, which we call the sample cumulant tensor, is generally not rate-optimal for $d\ge 3$, and we construct an estimator that attains the minimax rate. On the computational side, we study testing whether the $d$-th order cumulant tensor vanishes after whitening. We provide evidence, under a model-specific low-degree conjecture, that detection is computationally hard when $n\ll p^{d/2}$. At the same time, we identify a regime in which an efficiently computable estimator achieves error smaller than the separation at which polynomial-time tests can reliably distinguish the null from the alternative under the same conjecture. This reveals an unusual reverse detection--estimation gap: computationally efficient detection can be harder than computationally efficient estimation. The underlying reason is that the relevant loss, tensor spectral norm, is itself NP-hard to compute, creating a new form of computational--statistical gap.

math.ST↗
arXiv2026

Debiased parametric inference for regime-switching jump diffusions with infinite variation jumps

We study high-frequency inference for an ergodic regime-switching jump diffusion with locally stable infinite-variation jumps. The state and the finite-state regime are observed on the same grid. The mesh satisfies \(h_n\to0\), while the horizon satisfies \(T_n=nh_n\to\infty\). We estimate structural drift and diffusion parameters in the presence of unknown regime-specific Lévy measures. For a common activity index \(1<β<8/5\), we remove two leading jump biases from each regime's truncated variation. We then combine the diffusion estimator with estimated signed tail means in an adaptive drift contrast. We prove joint asymptotic normality at rates \(\sqrt{T_n}\) and \(\sqrt n\), together with feasible studentization. The growing horizon requires uniform control of the debiasing remainders and random denominators over increasing regime exposures. It also produces an explicit Lévy-measure term in the drift covariance. A fixed-grid median stabilizes the two-step correction without changing its first-order limit. We study finite-sample behavior by simulation and apply the method to 127.5 hours of cattle sensor data. In the application, jaw-activity labels are recorded by a device separate from the sensor that measures the state process.

math.ST↗
arXiv2026

Symmetries and Causality: Causal Effect Identification Beyond IID Data

We develop theory for the identification of complex causal queries from non-IID data. We propose a formal description of stochastic systems based on symmetries in data leaving causal mechanisms invariant. These symmetries become the source of statistical repetition for identification and mandate knowledge-transfer from observations to queries. The result is an abstract, simple and very general language for causal reasoning. We give descriptions of models and queries, and the formal infrastructure for their mathematically rigorous identification. The proposed structure enables a number of simplifying principles: A shared description of mechanisms and interventions, a duality of observed and queried structure, and the sub-division of the identification task into three steps - extraction, computation and query-expression. Our approach reproduces and matches standard theoretical results on IID data and transport of experimental and non-experimental data. By design, it additionally applies beyond IID data and captures complex causal queries not expressible via do- or soft-interventions.

math.ST↗