arXiv Science⌕ Search

arXiv subjects

Ben Adcock

Publications and source records attributed to Ben Adcock.

At least 19 recordsLinked to original sources
arXiv2026

The Sample Complexity of Learning Lipschitz Operators with respect to Gaussian Measures

Operator learning, the approximation of mappings between infinite-dimensional function spaces using machine learning, has gained increasing research attention in recent years. Operator approximations can serve as efficient surrogate models for problems in computational science and engineering, complementing traditional methods. However, despite their empirical success, our understanding of the underlying mathematical theory is in large part still incomplete. In this paper, we study the approximation of Lipschitz operators with respect to Gaussian measures. We prove higher Gaussian Sobolev regularity of Lipschitz operators and establish lower and upper bounds on the Hermite polynomial approximation error. We then study general reconstruction strategies of Lipschitz operators from $m$ arbitrary (potentially adaptive) linear samples. As a key finding, we tightly characterize the corresponding sample complexity, that is, the smallest achievable worst-case error among all possible choices of (adaptive) sampling and reconstruction strategies, in terms of $m$. As a consequence, we identify an inherent curse of sample complexity: No method to approximate Lipschitz operators based on $m$ linear samples can achieve algebraic convergence rates in $m$. On the positive side, we prove that a sufficiently fast spectral decay of the covariance operator of the underlying Gaussian measure guarantees convergence rates which are arbitrarily close to any algebraic rate. Overall, by tightly characterizing the sample complexity, our work confirms the intrinsic difficulty of learning Lipschitz operators, regardless of the data or learning technique.

cs.LG↗
arXiv2026

1-Lipschitz Neural Networks on Hadamard Manifolds

Controlling the Lipschitz constant of a neural network is a standard way to promote robustness and stability. Most existing constraining strategies are designed for Euclidean spaces. In this work, we construct and analyze a class of 1-Lipschitz neural networks on Hadamard manifolds. Our layers are of gradient-descent type, $1$-Lipschitz, and quasi-$α$-firmly nonexpansive. The core building blocks of the proposed architecture are Busemann functions, and we exploit the properties of Busemann gradient flows to design $1$-Lipschitz geometry-preserving layers. We provide explicit constructions and examples for hyperbolic manifolds and the manifold of symmetric positive definite (SPD) matrices. We test the proposed architecture in two numerical experiments: robust classification on the Poincaré disk and masked-Wishart covariance reconstruction. On the Poincaré disk, the proposed networks yield robust classifiers under hyperbolic perturbations. On the SPD manifold, we train SPD-valued denoisers and adopt them as a Plug-and-Play prior for a masked-Wishart covariance reconstruction problem. We show improved results from the nonexpansive denoiser over static, data-only, and Log-Euclidean denoising baselines, and empirically test its convergence properties.

math.NA↗
arXiv2026

Fourier extensions for matrix-function block encodings with error-independent subnormalization bounds

Block encodings of non-unitary matrix functions are central to quantum numerical linear algebra. Hamiltonian simulation is a natural input model for Hermitian matrices, but accurate block encodings often incur large subnormalization. We decouple the accuracy from the subnormalization by formulating the matrix-function block encoding as a Fourier-extension approximation problem, yielding a linear combination of unitaries for Hermitian matrix inputs. Fourier extensions approximate non-periodic functions by a Fourier series on a larger periodic domain, creating redundant coefficients that can be optimized for their absolute sum, and hence subnormalization. The coefficients may be chosen for optimal subnormalization with algebraic convergence, by tuning the subnormalization bound for increasing rates of exponential convergence, or by Sobolev-regularized fitting to accommodate more general spectral sets. Fourier-extension block encodings apply to eigenvalue transforms of Hermitian matrices, or to odd singular-value transforms of general matrices, including as a quantum linear systems algorithm.

quant-ph↗
arXiv2026

Near-Optimal Learning of Gaussian Sobolev Operators

A key question in operator learning is how to design surrogate operators with provable approximation guarantees in reasonable computational time. Whereas smooth operators can be approximated efficiently, i.e., with at least algebraic convergence in the amount of training data, learning finitely regular operators is known to be less efficient. The reason is an intrinsic curse of sample complexity, which allows only subalgebraic sample complexity rates. This fact makes it all the more important to develop algorithms which provably achieve these rates. In this work, we present a fully data-driven algorithm, termed Hermite-PCA approximation, for learning Gaussian Sobolev operators with near-optimal sample complexity. It employs principal component analysis and weighted least-squares methods and is therefore computationally efficient. Moreover, it is spectral, in the sense that it achieves faster (and near-optimal) convergence the higher the Sobolev regularity. We provide a full error analysis of this algorithm, taking into account all sources of error, along with numerical experiments that verify our theoretical results and empirically confirm the efficacy of Hermite-PCA approximation for learning Sobolev operators.

math.NA↗
arXiv2026

The devil in the (de)tails: an improved recovery guarantee for sparse approximation

Many functions exhibit approximate sparsity in their coefficients with respect to a given dictionary. In recent literature, sparse approximation in such a dictionary from i.i.d. pointwise samples, underpinned by compressed sensing, has become a powerful tool for high-dimensional function approximation. A key step in this framework is truncating the (typically countably-infinite) dictionary to a finite index set of size $n$, so that compressed sensing tools can be used to approximate the function by a sparse combination of these truncated dictionary elements. This introduces a discrete $L^2$-truncation error over the sample points, which in standard approaches, is bounded by the continuous $L^\infty$-norm. Such a deterministic, worst-case bound ignores the randomness of the sample points entirely. As a result, $n$ must be taken unnecessarily large to keep the truncation error under control, which directly inflates the size of the matrix involved in the sparse recovery algorithm and increases computational cost. In this paper, we show that by exploiting the i.i.d. structure of the sample points, the discrete $L^2$ truncation error admits a bound that instead reflects the faster decay behaviour of the continuous $L^2$-norm truncation error and yields significantly smaller truncation sets and decreased computational cost. We demonstrate this through applications to weighted Wiener spaces and anisotropic Sobolev spaces, in each case obtaining significantly smaller truncation sets than recent works. In addition, we also present an improved bound of independent interest for sparse approximation in bounded Riesz systems, where the measurement condition exhibits a smaller (and scale-invariant) dependence on the Riesz constants than in previous works.

math.NA↗
arXiv2026

Hybrid least squares for learning functions from highly noisy data

Motivated by the need for efficient estimation of conditional expectations, we consider a least-squares function approximation problem with heavily polluted data. Existing methods that are effective in the small-noise regime are suboptimal when large noise is present. To address this issue, we propose a hybrid approach that combines Christoffel sampling with optimal experimental design. We show that the proposed algorithm enjoys appropriate optimality properties for both sample point generation and noise mollification, leading to improved computational efficiency and sample complexity compared to existing methods. We also extend the algorithm to convexity-constrained settings with similar theoretical guarantees. When the target function is defined as the expectation of a random field, we further extend our approach to leverage adaptive random subspaces and establish results on the approximation capacity of the adaptive procedure. Our theoretical findings are supported by numerical studies on both synthetic data and on a more challenging stochastic simulation problem in computational finance.

stat.ML↗
arXiv2026

Christoffel-DPS: Optimal sensor placement in diffusion posterior sampling for arbitrary distributions

State estimation is a critical task in scientific, engineering and control applications. Since the reliability of reconstructions depends on the number and position of sensors, optimal sensor placement (OSP) is essential in scenarios where measurements are sparse and expensive. Classical OSP approaches rely on Gaussian assumptions and are consequently unable to account for the complex distributions encountered in many real-world systems. Generative-model-based reconstruction using sensor guided diffusion posterior sampling (DPS) has emerged as a promising technique for reconstructing states from highly complex distributions. However, existing sensor-selection methods either require unrealistically many sensors or emulate classical OSP, creating a mismatch between modern recovery models with classical OSP tools motivating the need for fundamentally new ideas towards OSP that match the recent advances made in powerful recovery models. We introduce a distribution-free sensor placement framework based on the Christoffel function: a mathematical formulation of optimal sampling and recovery guarantees for posterior sampling with arbitrary sensors and signal distributions, from which we derive a new OSP strategy with non-asymptotic bounds on the number of sensors needed for recovery. We develop Christoffel-DPS, with offline and online variants, instantiating Christoffel sampling for generative models. Christoffel-DPS outperforms Gaussian OSP baselines and existing generative-model placement methods, validating that distribution-free sensing is both theoretically principled and practically superior. The framework is model-agnostic; we demonstrate its application to a range of unconditional DPS and flow-matching models on structurally non-Gaussian benchmarks, showing the efficacy of Christoffel-DPS in low sensor budget regimes.

cs.LG↗
arXiv2026

GRIFDIR: Graph Resolution-Invariant FEM Diffusion Models in Function Spaces over Irregular Domains

Score-based diffusion models in infinite-dimensional function spaces provide a mathematically principled framework for modelling function-valued data, offering key advantages such as resolution invariance and the ability to handle irregular discretisations. However, practical implementations have struggled to fully realise these benefits. Existing backbones like Fourier neural operators are often biased towards regular grids and fail to generalise to complex domain topologies. We propose a novel architecture for function-space diffusion models that represents generalised graph convolutional kernels as finite element functions, enabling the model to naturally handle unstructured meshes and complex geometries. We demonstrate the efficacy of our network architecture through a series of unconditional and conditional sampling experiments across diverse geometries, including non-convex and multiply-connected domains. Our results show that the proposed method maintains resolution invariance and achieves high fidelity in capturing functional distributions on non-trivial geometries.

cs.LG↗
arXiv2026

Universal, sample-optimal algorithms for recovery of anisotropic functions from i.i.d. samples

A key problem in approximation theory is the recovery of high-dimensional functions from samples. In many cases, the functions of interest exhibit anisotropic smoothness, and, in many practical settings, the nature of this anisotropy may be unknown a priori. Therefore, an important question involves the development of universal algorithms, namely, algorithms that simultaneously achieve optimal or near-optimal rates of convergence across a range of different anisotropic smoothness classes. In this work, we consider universal approximation of periodic functions that belong to anisotropic Sobolev spaces and anisotropic dominating mixed smoothness Sobolev spaces. Our first result is the construction of a universal algorithm. This recasts function recovery as a sparse recovery problem for Fourier coefficients and then exploits compressed sensing to yield the desired approximation rates. Note that this algorithm is nonadaptive, as it does not seek to learn the anisotropic smoothness of the target function. We then demonstrate optimality of this algorithm up to a dimension-independent polylogarithmic factor. We do this by presenting a lower bound for the adaptive $m$-width for the unit balls of such function classes. Finally, we demonstrate the necessity of nonlinear algorithms. We show that universal linear algorithms can achieve rates that are at best suboptimal by a dimension-dependent polylogarithmic factor. In other words, they suffer from a curse of dimensionality in the rate -- a phenomenon which justifies the necessity of nonlinear algorithms for universal recovery.

math.NA↗
arXiv2026

Christoffel Adaptive Sampling for Sparse Random Feature Expansions

Random Feature Models (RFMs) have become a powerful tool for approximating multivariate functions and solving partial differential equations efficiently. Sparse Random Feature Expansions (SRFE) improve traditional RFMs by incorporating sparsity, making it particularly effective in data-scarce settings. In this work, we integrate active learning with sparse random feature approximations to improve sampling efficiency. Specifically, we incorporate the Christoffel function to guide an adaptive sampling process, dynamically selecting informative sample points based on their contribution to the function space. This approach optimizes the distribution of sample points by leveraging the Christoffel function associated with an iteratively-chosen basis obtained by the sparse recovery solver. We conduct numerical experiments comparing adaptive and nonadaptive sampling strategies with the SRFE framework and examine their accuracy for various function approximation tasks. Overall, our results demonstrate the advantages of adaptive sampling in maintaining high accuracy while reducing sample complexity for SRFE, highlighting its potential for scientific computing tasks where data is expensive to acquire.

math.NA↗
arXiv2026

A unified framework for learning with nonlinear model classes from arbitrary linear samples

We study the fundamental problem of learning an unknown object from data using a prescribed model class. We introduce a unified framework that accommodates objects in arbitrary Hilbert spaces, general (possibly vector-valued) random linear measurements and general types of nonlinear models. We establish novel learning guarantees for this framework that explicitly relate the required amount of data to structural properties of the model class, yielding near-optimal generalization bounds. A central concept we introduce is the variation of a model class relative to a distribution of sampling operators, which quantifies how the model interacts with the measurement process. Combined with entropy integrals that capture the model's complexity, this forms the foundation of our guarantees. Our framework is sufficiently general to recover and unify various well-known problems, such as matrix sketching, compressed sensing with isotropic measurements and compressed sensing with generative models. In each case, existing results arise as direct corollaries of our theory. For compressed sensing with generative models, we also derive the first guarantees for arbitrary Lipschitz generative maps combined with general linear measurements. Overall, our work provides a unified perspective on learning from general data and introduces novel theoretical guarantees that consolidate, sharpen and extend existing results.

cs.LG↗
arXiv2026

Towards Sharp Minimax Risk Bounds for Operator Learning

We develop a minimax theory for operator learning, where the goal is to estimate an unknown operator between separable Hilbert spaces from finitely many noisy input-output samples. For uniformly bounded Lipschitz operators, we prove information-theoretic lower bounds together with matching or near-matching upper bounds, covering both fixed and random designs under Hilbert-valued Gaussian noise and Gaussian white noise errors. The rates are controlled by the spectrum of the covariance operator of the measure that defines the error metric. Our setup is very general and allows for measures with unbounded support. A key implication is a curse of sample complexity, which shows that the minimax risk for generic Lipschitz operators cannot decay at any algebraic rate in the sample size. We obtain sharp characterizations when the covariance spectrum decays exponentially and provide general upper and lower bounds in slower-decay regimes. Finally, we show that assuming higher regularity, i.e., Hölder smoothness, does not improve minimax rates over the Lipschitz case, up to potential constants. Thus, we show that learning operators of any finite regularity necessarily suffers a curse of sample complexity.

math.ST↗
arXiv2025

Refinement-based Christoffel sampling for least squares approximation in non-orthogonal bases

We introduce a refinement-based Christoffel sampling (RCS) algorithm for least squares approximation in the span of a given, generally non-orthogonal set of functions $Φ_n = \{ϕ_1, \dots, ϕ_n\}$. A standard sampling strategy for this problem is Christoffel sampling, which achieves near-best approximations in probability using only $\mathcal{O}(n \log(n))$ samples. However, it requires i.i.d. sampling from a distribution whose density is proportional to the inverse Christoffel function $k_n$, the computation of which requires an orthonormal basis. As a result, existing approaches for non-orthogonal bases $Φ_n$ typically rely on costly discrete orthogonalization. We propose a new iterative algorithm, inspired by recent advances in approximate leverage score sampling, that avoids this bottleneck. Crucially, while the computational cost of discrete orthogonalization grows proportionally with $\|k_n\|_{L^\infty(X)}$, the cost of our approach increases only logarithmically in $\|k_n\|_{L^\infty(X)}$. In addition, we account for finite-precision effects by considering a numerical variant of the Christoffel function, ensuring that the algorithm relies only on computable quantities. Alongside a convergence proof, we present extensive numerical experiments demonstrating the efficiency and robustness of the proposed method.

math.NA↗
arXiv2025

How many measurements are enough? Bayesian recovery in inverse problems with general distributions

We study the sample complexity of Bayesian recovery for solving inverse problems with general prior, forward operator and noise distributions. We consider posterior sampling according to an approximate prior $\mathcal{P}$, and establish sufficient conditions for stable and accurate recovery with high probability. Our main result is a non-asymptotic bound that shows that the sample complexity depends on (i) the intrinsic complexity of $\mathcal{P}$, quantified by its so-called approximate covering number, and (ii) concentration bounds for the forward operator and noise distributions. As a key application, we specialize to generative priors, where $\mathcal{P}$ is the pushforward of a latent distribution via a Deep Neural Network (DNN). We show that the sample complexity scales log-linearly with the latent dimension $k$, thus establishing the efficacy of DNN-based priors. Generalizing existing results on deterministic (i.e., non-Bayesian) recovery for the important problem of random sampling with an orthogonal matrix $U$, we show how the sample complexity is determined by the coherence of $U$ with respect to the support of $\mathcal{P}$. Hence, we establish that coherence plays a fundamental role in Bayesian recovery as well. Overall, our framework unifies and extends prior work, providing rigorous guarantees for the sample complexity of solving Bayesian inverse problems with arbitrary distributions.

cs.LG↗
arXiv2025

Optimal sampling for least-squares approximation

Least-squares approximation is one of the most important methods for recovering an unknown function from data. While in many applications the data is fixed, in many others there is substantial freedom to choose where to sample. In this paper, we review recent progress on near-optimal random sampling strategies for (weighted) least-squares approximation in arbitrary linear spaces. We introduce the Christoffel function as a key quantity in the analysis of (weighted) least-squares approximation from random samples, then show how it can be used to construct a random sampling strategy, termed Christoffel sampling, that possesses near-optimal sample complexity: namely, the number of samples scales log-linearly in the dimension of the approximation space $n$. We discuss a series of variations, extensions and further topics, and throughout highlight connections to approximation theory, machine learning, information-based complexity and numerical linear algebra. Finally, motivated by various contemporary applications, we consider a generalization of the classical setting where the samples need not be pointwise samples of a scalar-valued function, and the approximation space need not be linear. We show that, even in this significantly more general setting, suitable generalizations of Christoffel function still determine the sample complexity. Consequently, these can be used to design enhanced, Christoffel sampling strategies in a unified way for general recovery problems. This article is largely self-contained, and intended to be accessible to nonspecialists.

stat.ML↗
arXiv2025

Function recovery and optimal sampling in the presence of nonuniform evaluation costs

We consider recovering a function $f : D \rightarrow \mathbb{C}$ in an $n$-dimensional linear subspace $\mathcal{P}$ from i.i.d. pointwise samples via (weighted) least-squares estimators. Different from most works, we assume the cost of evaluating $f$ is potentially nonuniform, and governed by a cost function $c : D \rightarrow (0,\infty)$ which may blow up at certain points. We therefore strive to choose the sampling measure in a way that minimizes the expected total cost. We provide a recovery guarantee which asserts accurate and stable recovery with an expected cost depending on the Christoffel function and Remez constant of the space $\mathcal{P}$. This leads to a general recipe for finding a good sampling measure for general $c$. As an example, we consider one-dimensional polynomial spaces. Here, we provide two strategies for choosing the sampling measure, which we prove are optimal (up to constants and log factors) in the case of algebraically-growing cost functions.

math.NA↗
arXiv2025

Optimal approximation of infinite-dimensional holomorphic functions II: recovery from i.i.d. pointwise samples

Infinite-dimensional, holomorphic functions have been studied in detail over the last several decades, due to their relevance to parametric differential equations and computational uncertainty quantification. The approximation of such functions from finitely many samples is of particular interest, due to the practical importance of constructing surrogate models to complex mathematical models of physical processes. In a previous work, [6] we studied the approximation of so-called Banach-valued, $(\boldsymbol{b},\varepsilon)$-holomorphic functions on the infinite-dimensional hypercube $[-1,1]^{\mathbb{N}}$ from $m$ (potentially adaptive) samples. In particular, we derived lower bounds for the adaptive $m$-widths for classes of such functions, which showed that certain algebraic rates of the form $m^{1/2-1/p}$ are the best possible regardless of the sampling-recovery pair. In this work, we continue this investigation by focusing on the practical case where the samples are pointwise evaluations drawn identically and independently from a probability measure. Specifically, for Hilbert-valued $(\boldsymbol{b},\varepsilon)$-holomorphic functions, we show that the same rates can be achieved (up to a small polylogarithmic or algebraic factor) for essentially arbitrary tensor-product Jacobi (ultraspherical) measures. Our reconstruction maps are based on least squares and compressed sensing procedures using the corresponding orthonormal Jacobi polynomials. In doing so, we strengthen and generalize past work that has derived weaker nonuniform guarantees for the uniform and Chebyshev measures (and corresponding polynomials) only. We also extend various best $s$-term polynomial approximation error bounds to arbitrary Jacobi polynomial expansions. Overall, we demonstrate that i.i.d.\ pointwise samples are near-optimal for the recovery of infinite-dimensional, holomorphic functions.

math.NA↗
arXiv2024

Optimal deep learning of holomorphic operators between Banach spaces

Operator learning problems arise in many key areas of scientific computing where Partial Differential Equations (PDEs) are used to model physical systems. In such scenarios, the operators map between Banach or Hilbert spaces. In this work, we tackle the problem of learning operators between Banach spaces, in contrast to the vast majority of past works considering only Hilbert spaces. We focus on learning holomorphic operators - an important class of problems with many applications. We combine arbitrary approximate encoders and decoders with standard feedforward Deep Neural Network (DNN) architectures - specifically, those with constant width exceeding the depth - under standard $\ell^2$-loss minimization. We first identify a family of DNNs such that the resulting Deep Learning (DL) procedure achieves optimal generalization bounds for such operators. For standard fully-connected architectures, we then show that there are uncountably many minimizers of the training problem that yield equivalent optimal performance. The DNN architectures we consider are `problem agnostic', with width and depth only depending on the amount of training data $m$ and not on regularity assumptions of the target operator. Next, we show that DL is optimal for this problem: no recovery procedure can surpass these generalization bounds up to log terms. Finally, we present numerical results demonstrating the practical performance on challenging problems including the parametric diffusion, Navier-Stokes-Brinkman and Boussinesq PDEs.

cs.LG↗