arXiv subjects

Aaditya Ramdas

Publications and source records attributed to Aaditya Ramdas.

At least 109 records · Page 6Linked to original sources

arXiv2025

Sharp Matrix Empirical Bernstein Inequalities

We present two sharp, closed-form empirical Bernstein inequalities for symmetric random matrices with bounded eigenvalues. By sharp, we mean that both inequalities adapt to the unknown variance in a tight manner: the deviation captured by the first-order $1/\sqrt{n}$ term asymptotically matches the matrix Bernstein inequality exactly, including constants, the latter requiring knowledge of the variance. Our first inequality holds for the sample mean of independent matrices, and our second inequality holds for a mean estimator under martingale dependence at stopping times.

math.PR↗

arXiv2025

A Permutation-free Kernel Two-Sample Test

The kernel Maximum Mean Discrepancy~(MMD) is a popular multivariate distance metric between distributions that has found utility in two-sample testing. The usual kernel-MMD test statistic is a degenerate U-statistic under the null, and thus it has an intractable limiting distribution. Hence, to design a level-$α$ test, one usually selects the rejection threshold as the $(1-α)$-quantile of the permutation distribution. The resulting nonparametric test has finite-sample validity but suffers from large computational cost, since every permutation takes quadratic time. We propose the cross-MMD, a new quadratic-time MMD test statistic based on sample-splitting and studentization. We prove that under mild assumptions, the cross-MMD has a limiting standard Gaussian distribution under the null. Importantly, we also show that the resulting test is consistent against any fixed alternative, and when using the Gaussian kernel, it has minimax rate-optimal power against local alternatives. For large sample sizes, our new cross-MMD provides a significant speedup over the MMD, for only a slight loss in power.

stat.ME↗

arXiv2025

Hypothesis testing with e-values

This book is written to offer a humble, but unified, treatment of e-values in hypothesis testing. It is organized into three parts: Fundamental Concepts, Core Ideas, and Advanced Topics. The first part includes four chapters that introduce the basic concepts. The second part includes five chapters of core ideas such as universal inference, log-optimality, e-processes, operations on e-values, and e-values in multiple testing. The third part contains seven chapters of advanced topics. The book collates important results from a variety of modern papers on e-values and related concepts, and also contains many results not published elsewhere. It offers a coherent and comprehensive picture on a fast-growing research area, and is ready to use as the basis of a graduate course in statistics and related fields.

math.ST↗

arXiv2025

Bringing closure to FDR control: beating the e-Benjamini-Hochberg procedure

False discovery rate (FDR) has been a key metric for error control in multiple hypothesis testing, and many methods have developed for FDR control across a diverse cross-section of settings and applications. We develop a closure principle for all FDR controlling procedures, i.e., we provide a characterization based on e-values for all admissible FDR controlling procedures. A general version of this closure principle can recover any multiple testing error metric and allows one to choose the error metric post-hoc. We leverage this idea to formulate the closed eBH procedure, a (usually strict) improvement over the eBH procedure for FDR control when provided with e-values. This also yields a closed BY procedure that dominates the Benjamini-Yekutieli (BY) procedure for FDR control with arbitrarily dependent p-values, thus proving that the latter is inadmissibile. We demonstrate the practical performance of our new procedures in simulations.

stat.ME↗

arXiv2025

Quantum Sequential Universal Hypothesis Testing

Quantum hypothesis testing (QHT) concerns the statistical inference of unknown quantum states. In the general setting of composite hypotheses, the goal of QHT is to determine whether an unknown quantum state belongs to one or another of two classes of states based on the measurement of a number of copies of the state. Prior art on QHT with composite hypotheses focused on a fixed-copy two-step protocol, with state estimation followed by an optimized joint measurement. However, this fixed-copy approach may be inefficient, using the same number of copies irrespective of the inherent difficulty of the testing task. To address these limitations, we introduce the quantum sequential universal test (QSUT), a novel framework for sequential QHT in the general case of composite hypotheses. QSUT builds on universal inference, and it alternates between adaptive local measurements aimed at exploring the hypothesis space and joint measurements optimized for maximal discrimination. QSUT is proven to rigorously control the type I error under minimal assumptions about the hypothesis structure. We present two practical instantiations of QSUT, one based on the Helstrom-Holevo test and one leveraging shallow variational quantum circuits. Empirical results across a range of composite QHT tasks demonstrate that QSUT consistently reduces copy complexity relative to state-of-the-art fixed-copy strategies.

quant-ph↗

arXiv2025

Multiple testing with anytime-valid Monte Carlo p-values

In contemporary problems involving genetic or neuroimaging data, thousands of hypotheses need to be tested. Due to their high power, and finite sample guarantees on type-I error under weak assumptions, Monte Carlo permutation tests are often considered as gold standard for these settings. However, the enormous computational effort required for (thousands of) permutation tests is a major burden. In this paper, we integrate recently constructed anytime-valid permutation p-values into a broad class of multiple testing procedures, including the Benjamini-Hochberg procedure. This allows to fully adapt the number of permutations to the underlying data and thus, for example, to the number of rejections made by the multiple testing procedure. Even though this data-adaptive stopping can induce dependencies between the p-values that violate the usual assumptions of the Benjamini-Hochberg procedure, we prove that our approach controls the false discovery rate under mild assumptions. Furthermore, our method provably decreases the required number of permutations substantially without compromising power. On a real genomics data set, our method reduced the computational time from more than three days to less than four minutes while increasing the number of rejections.

stat.ME↗

arXiv2025

Nonparametric Two-Sample Testing by Betting

We study the problem of designing consistent sequential two-sample tests in a nonparametric setting. Guided by the principle of testing by betting, we reframe this task into that of selecting a sequence of payoff functions that maximize the wealth of a fictitious bettor, betting against the null in a repeated game. In this setting, the relative increase in the bettor's wealth has a precise interpretation as the measure of evidence against the null, and thus our sequential test rejects the null when the wealth crosses an appropriate threshold. We develop a general framework for setting up the betting game for two-sample testing, in which the payoffs are selected by a prediction strategy as data-driven predictable estimates of the witness function associated with the variational representation of some statistical distance measures, such as integral probability metrics (IPMs). We then formally relate the statistical properties of the test~(such as consistency, type-II error exponent and expected sample size) to the regret of the corresponding prediction strategy. We construct a practical sequential two-sample test by instantiating our general strategy with the kernel-MMD metric, and demonstrate its ability to adapt to the difficulty of the unknown alternative through theoretical and empirical results. Our framework is versatile, and easily extends to other problems; we illustrate this by applying our approach to construct consistent tests for the following problems: (i) time-varying two-sample testing with non-exchangeable observations, and (ii) an abstract class of "invariant" testing problems, including symmetry and independence testing.

math.ST↗

arXiv2025

Anytime-valid FDR control with the stopped e-BH procedure

The recent e-Benjamini-Hochberg (e-BH) procedure for multiple hypothesis testing is known to control the false discovery rate (FDR) under arbitrary dependence between the input e-values. This paper points out an important subtlety when applying the e-BH procedure with e-processes, which are sequential generalizations of e-values (where the data are observed sequentially). Since adaptively stopped e-processes are e-values, the e-BH procedure can be repeatedly applied at every time step, and one can continuously monitor the e-processes and the rejection sets obtained. One would hope that the "stopped e-BH procedure" (se-BH) has an FDR guarantee for the rejection set obtained at any stopping time. However, while this is true if the data in different streams are independent, it is not true in full generality, because each stopped e-process is an e-value only for stopping times in its own local filtration, but the se-BH procedure employs a stopping time with respect to a global filtration. This can cause information to leak across time, allowing one stream to know its future by knowing past data of another stream. This paper formulates a simple causal condition under which local e-processes are also global e-processes and thus the se-BH procedure does indeed control the FDR. The condition excludes unobserved confounding from the past and is met under most reasonable scenarios including genomics.

stat.ME↗

arXiv2025

Density estimation with atoms, and functional estimation for mixed discrete-continuous data

In classical density (or density-functional) estimation, it is standard to assume that the underlying distribution has a density with respect to the Lebesgue measure. However, when the data distribution is a mixture of continuous and discrete components, the resulting methods are inconsistent in theory and perform poorly in practice. In this paper, we point out that a minor modification of existing methods for nonparametric density (functional) estimation can allow us to fully remove this assumption while retaining nearly identical theoretical guarantees and improved empirical performance. Our approach is very simple: data points that appear exactly once are likely to originate from the continuous component, whereas repeated observations are indicative of the discrete part. Leveraging this observation, we modify existing estimators for a broad class of functionals of the continuous component of the mixture; this modification is a "wrapper" in the sense that the user can use any underlying method of their choice for continuous density functional estimation. Our modifications deliver consistency without requiring knowledge of the discrete support, the mixing proportion, and without imposing additional assumptions beyond those needed in the absence of the discrete part. Thus, various theorems and existing software packages can be made automatically more robust, with absolutely no additional price when the data is not truly mixed.

stat.ME↗

arXiv2025

Asymptotic and compound e-values: multiple testing and empirical Bayes

We explicitly define the notions of (bona fide, approximate or asymptotic) compound p-values and e-values, which have been implicitly presented and used in the recent multiple testing literature. While it is known that the e-BH procedure with compound e-values controls the FDR, we show the converse: every FDR controlling procedure can be recovered by instantiating the e-BH procedure with certain compound e-values. Since compound e-values are closed under averaging, this allows for combination and derandomization of arbitrary FDR procedures. We then connect compound e-values to empirical Bayes. In particular, we use the fundamental theorem of compound decision theory to derive the log-optimal simple separable compound e-value for testing a set of point nulls against point alternatives: it is a ratio of mixture likelihoods. As one example, we construct asymptotic compound e-values for multiple t-tests, where the (nuisance) variances may be different across hypotheses. Our construction may be interpreted as a data-driven instantiation of the optimal discovery procedure, and our results provide the first type-I error guarantees for the same, along with significant power gains.

stat.ME↗

arXiv2025

Improving Wald's (approximate) sequential probability ratio test by avoiding overshoot

Wald's sequential probability ratio test (SPRT) is a cornerstone of sequential analysis. Based on desired type-I, II error levels $α, β$, it stops when the likelihood ratio crosses certain thresholds, guaranteeing optimality of the expected sample size. However, these thresholds are not closed form and the test is often applied with approximate thresholds $(1-β)/α$ and $β/(1-α)$ (approximate SPRT). When $β> 0$, this neither guarantees error control at $α,β$ nor optimality. When $β=0$ (power-one SPRT), this method is conservative and not optimal. The looseness in both cases is caused by \emph{overshoot}: the test statistic overshoots the thresholds at the stopping time. Numerically calculating thresholds may be infeasible, and most software packages do not do this. We improve the approximate SPRT by modifying the test statistic to avoid overshoot. Our `sequential boosting' technique uniformly improves power-one SPRTs $(β=0)$ for simple nulls and alternatives, or for one-sided nulls and alternatives in exponential families. When $β> 0$, our techniques provide guaranteed error control at $α,β$, while needing less samples than the approximate SPRT in our simulations. We also provide several nontrivial extensions: confidence sequences, sampling without replacement and conformal martingales.

stat.ME↗

arXiv2025

QuTE: decentralized multiple testing on sensor networks with false discovery rate control

This paper designs methods for decentralized multiple hypothesis testing on graphs that are equipped with provable guarantees on the false discovery rate (FDR). We consider the setting where distinct agents reside on the nodes of an undirected graph, and each agent possesses p-values corresponding to one or more hypotheses local to its node. Each agent must individually decide whether to reject one or more of its local hypotheses by only communicating with its neighbors, with the joint aim that the global FDR over the entire graph must be controlled at a predefined level. We propose a simple decentralized family of Query-Test-Exchange (QuTE) algorithms and prove that they can control FDR under independence or positive dependence of the p-values. Our algorithm reduces to the Benjamini-Hochberg (BH) algorithm when after graph-diameter rounds of communication, and to the Bonferroni procedure when no communication has occurred or the graph is empty. To avoid communicating real-valued p-values, we develop a quantized BH procedure, and extend it to a quantized QuTE procedure. QuTE works seamlessly in streaming data settings, where anytime-valid p-values may be continually updated at each node. Last, QuTE is robust to arbitrary dropping of packets, or a graph that changes at every step, making it particularly suitable to mobile sensor networks involving drones or other multi-agent systems. We study the power of our procedure using a simulation suite of different levels of connectivity and communication on a variety of graph structures, and also provide an illustrative real-world example.

stat.ME↗

arXiv2025

Improving the statistical efficiency of cross-conformal prediction

Vovk (2015) introduced cross-conformal prediction, a modification of split conformal designed to improve the width of prediction sets. The method, when trained with a miscoverage rate equal to $α$ and $n \gg K$, ensures a marginal coverage of at least $1 - 2α- 2(1-α)(K-1)/(n+K)$, where $n$ is the number of observations and $K$ denotes the number of folds. A simple modification of the method achieves coverage of at least $1-2α$. In this work, we propose new variants of both methods that yield smaller prediction sets without compromising the latter theoretical guarantees. The proposed methods are based on recent results deriving more statistically efficient combination of p-values that leverage exchangeability and randomization. Simulations confirm the theoretical findings and bring out some important tradeoffs.

stat.ML↗

arXiv2025

Sequential Kernelized Independence Testing

Independence testing is a classical statistical problem that has been extensively studied in the batch setting when one fixes the sample size before collecting data. However, practitioners often prefer procedures that adapt to the complexity of a problem at hand instead of setting sample size in advance. Ideally, such procedures should (a) stop earlier on easy tasks (and later on harder tasks), hence making better use of available resources, and (b) continuously monitor the data and efficiently incorporate statistical evidence after collecting new data, while controlling the false alarm rate. Classical batch tests are not tailored for streaming data: valid inference after data peeking requires correcting for multiple testing which results in low power. Following the principle of testing by betting, we design sequential kernelized independence tests that overcome such shortcomings. We exemplify our broad framework using bets inspired by kernelized dependence measures, e.g., the Hilbert-Schmidt independence criterion. Our test is also valid under non-i.i.d., time-varying settings. We demonstrate the power of our approaches on both simulated and real data.

stat.ML↗

arXiv2025

Auditing Fairness by Betting

We provide practical, efficient, and nonparametric methods for auditing the fairness of deployed classification and regression models. Whereas previous work relies on a fixed-sample size, our methods are sequential and allow for the continuous monitoring of incoming data, making them highly amenable to tracking the fairness of real-world systems. We also allow the data to be collected by a probabilistic policy as opposed to sampled uniformly from the population. This enables auditing to be conducted on data gathered for another purpose. Moreover, this policy may change over time and different policies may be used on different subpopulations. Finally, our methods can handle distribution shift resulting from either changes to the model or changes in the underlying population. Our approach is based on recent progress in anytime-valid inference and game-theoretic statistics-the "testing by betting" framework in particular. These connections ensure that our methods are interpretable, fast, and easy to implement. We demonstrate the efficacy of our approach on three benchmark fairness datasets.

stat.ML↗

arXiv2025

Time-Uniform Confidence Spheres for Means of Random Vectors

We study sequential mean estimation in $\mathbb{R}^d$. In particular, we derive time-uniform confidence spheres -- confidence sphere sequences (CSSs) -- which contain the mean of random vectors with high probability simultaneously across all sample sizes. Our results include a dimension-free CSS for log-concave random vectors, a dimension-free CSS for sub-Gaussian random vectors, and CSSs for sub-$ψ$ random vectors (which includes sub-gamma, sub-Poisson, and sub-exponential distributions). Many of our results are optimal. For sub-Gaussian distributions we also provide a CSS which tracks a time-varying mean, generalizing Robbins' mixture approach to the multivariate setting. Finally, we provide several CSSs for heavy-tailed random vectors (two moments only). Our bounds hold under a martingale assumption on the mean and do not require that the observations be iid. Our work is based on PAC-Bayesian theory and inspired by an approach of Catoni and Giulini.

math.ST↗

arXiv2025

Time-Uniform Self-Normalized Concentration for Vector-Valued Processes

Self-normalized processes arise naturally in many learning-related tasks. While self-normalized concentration has been extensively studied for scalar-valued processes, there are few results for multidimensional processes outside of the sub-Gaussian setting. In this work, we construct a general, self-normalized inequality for multivariate processes that satisfy a simple yet broad sub-$ψ$ tail condition, which generalizes assumptions based on cumulant generating functions. From this general inequality, we derive an upper law of the iterated logarithm for sub-$ψ$ vector-valued processes, which is tight up to small constants. We show how our inequality can be leveraged to derive a variety of novel, self-normalized concentration inequalities under both light and heavy-tailed observations. Further, we provide applications in prototypical statistical tasks, such as parameter estimation in online linear regression, autoregressive modeling, and bounded mean estimation via a new (multivariate) empirical Bernstein concentration inequality.

math.PR↗

arXiv2025

Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedback

In interactive imitation learning (IL), uncertainty quantification offers a way for the learner (i.e. robot) to contend with distribution shifts encountered during deployment by actively seeking additional feedback from an expert (i.e. human) online. Prior works use mechanisms like ensemble disagreement or Monte Carlo dropout to quantify when black-box IL policies are uncertain; however, these approaches can lead to overconfident estimates when faced with deployment-time distribution shifts. Instead, we contend that we need uncertainty quantification algorithms that can leverage the expert human feedback received during deployment time to adapt the robot's uncertainty online. To tackle this, we draw upon online conformal prediction, a distribution-free method for constructing prediction intervals online given a stream of ground-truth labels. Human labels, however, are intermittent in the interactive IL setting. Thus, from the conformal prediction side, we introduce a novel uncertainty quantification algorithm called intermittent quantile tracking (IQT) that leverages a probabilistic model of intermittent labels, maintains asymptotic coverage guarantees, and empirically achieves desired coverage levels. From the interactive IL side, we develop ConformalDAgger, a new approach wherein the robot uses prediction intervals calibrated by IQT as a reliable measure of deployment-time uncertainty to actively query for more expert feedback. We compare ConformalDAgger to prior uncertainty-aware DAgger methods in scenarios where the distribution shift is (and isn't) present because of changes in the expert's policy. We find that in simulated and hardware deployments on a 7DOF robotic manipulator, ConformalDAgger detects high uncertainty when the expert shifts and increases the number of interventions compared to baselines, allowing the robot to more quickly learn the new behavior.

cs.RO↗