arXiv Science⌕ Search

arXiv subjects

Alex Townsend

Publications and source records attributed to Alex Townsend.

At least 19 recordsLinked to original sources
arXiv2026

A Complete Resolution of Forsythe's Conjecture for Restarted Conjugate Gradients

Forsythe's conjecture, published in 1968, asserts that for each restart length $s$, every exact-arithmetic restarted conjugate-gradient iteration on a real symmetric positive definite problem either terminates or has normalised residuals that converge separately along the even and odd restart subsequences. Apart from the classical steepest-descent case, this asymptotic question remained unresolved in full generality for nearly six decades. We give a complete classification by restart length in the original finite-dimensional setting and identify a sharp threshold. For $s=2$ and $s=3$, every problem either terminates or has convergent even and odd residual directions. For every $s\ge4$, there is a diagonal positive definite counterexample of dimension $s+4$ which never terminates and whose even residual directions do not converge. Together with Akaike's theorem for $s=1$, this shows that the conjectured universal conclusion is true precisely for $s\in\{1,2,3\}$ and false for every $s\ge4$. The positive results follow from a degree-independent double-orthogonality identity and an analysis of the low-degree fixed-point sets. At restart length four, rational interval arithmetic and Sturm sequences certify a transverse Hopf point of the leading vector field of the rescaled squared-weight map. Analytic periodic-orbit and shadowing arguments yield the counterexample at restart length four, and degree elevation extends the construction to every larger restart length. The classification for all $s\ge2$ is also formally verified in Lean.

math.NA↗
arXiv2026

A zero-one law for one-shot system identification

Can a model be identified from one experiment? We study analytic systems that are linearly parameterized by a combination of prescribed dictionary terms, such as partial differential operators and dynamical systems. For a single input-response pair, recovery is possible exactly when the evaluated dictionary terms are linearly independent. We prove a sharp zero-one law: either no input uniquely determines the coefficients, or almost every random input sampled from a nondegenerate Gaussian measure does. This dichotomy reduces one-shot system identification to a question about degenerate inputs and provides an a posteriori certificate for any recovered model. Numerical examples recover dynamical systems, nonlinear partial differential equations, and structured matrix families from single trajectory data, while also detecting when an extra probe is necessary.

math.NA↗
arXiv2026

Blow-up Parameter Landscapes for Polynomial Dynamical Systems

Finite-time blow-up is one of the ways in which a dynamical model can become singular, often signaling the breakdown of either the modeled physical system or the model itself. Determining whether blow-up occurs, and for which parameter values and initial conditions, is therefore a fundamental problem in the analysis of nonlinear dynamical systems. We develop a numerical framework for identifying regions of parameter space in which a dynamical system governed by a system of first-order ordinary differential equations with polynomial right-hand sides exhibits finite-time blow-up for at least one initial condition. The approach combines compactification of the phase space with computational algebraic techniques, producing partitioned parameter landscapes that reveal blow-up and non-blow-up regimes. Through several examples, we show that the method replaces problem-specific hand calculations with an automated computational tool for analyzing blow-up regions in parameter-dependent dynamical systems.

math.DS↗
arXiv2026

Transpose-free linear algebra

We study the limitations of matrix-free algorithms that access a matrix $A$ only through forward matrix-vector products (matvecs) $x \mapsto Ax$, without access to the transpose $A^\top$ or its action. This setting arises naturally in operator learning, inverse problems, and matrix-free PDE solvers, where adjoint evaluations may be unavailable or prohibitively expensive. We show that the lack of transpose access creates severe and sometimes insurmountable theoretical barriers. For Krylov methods, we prove that the sequence of projected operator norms produced by Arnoldi iteration can follow any prescribed nondecreasing curve, showing that forward matvecs alone provide essentially no reliable information about the spectral norm. For several core problems, including least squares, norm estimation, column subset selection, and local maximum volume, we establish non-identifiability results; distinct matrices can generate identical forward-query transcripts while having fundamentally different solutions. We also prove quantitative lower bounds on the number of forward matvecs required for approximation tasks. In particular, any algorithm that computes a near-optimal rank-$k$ approximation must use at least $n$ queries, and estimating the Frobenius norm to relative accuracy $\eps$ requires $\Omega(\eps^{-2})$ queries when $n$ is sufficiently large, matching the complexity of Hutchinson-type estimators up to constants. Although some problems remain solvable without transpose access, the transpose-free setting is fundamentally more limited in both identifiability and efficiency.

math.NA↗
arXiv2026

Oblivious Subspace Injection Is Not Enough for Relative Error

Oblivious subspace injection (OSI) was introduced by Cama\~no, Epperly, Meyer, and Tropp in 2025 as a much weaker sketching property than oblivious subspace embedding (OSE) that still yields constant-factor guarantees for randomized low-rank approximation and sketch-and-solve least-squares regression. At the Simons Institute in Berkeley during a workshop in October 2025, it was asked whether OSIs also imply relative error bounds rather than just constant-factor guarantees. We show that, from a theoretical standpoint, OSI alone does not yield OSE-style relative-error guarantees whose failure probability is controlled solely by the OSI failure parameter, even though OSI sketches often perform extremely well in practice. We provide counterexamples showing this for sketch-and-solve least squares and for randomized SVD in the Frobenius norm. The missing ingredient from a sketch satisfying only OSI is upper control on the optimal residual or tail component, and when one ensures the sketch has this additional property, a near-relative-error bound is recovered. We also show that there is a natural $\ell_p$ analogue of OSI giving constant-factor sketch-and-solve bounds.

math.NA↗
arXiv2026

Beyond singular value gaps in randomized subspace approximation

The success of randomized range finders (RRFs) is typically analyzed via the singular value gaps of a target matrix $A$. In this work, we show that the so-called Frobenius singular value ratio provides a sharper analysis of an RRF's subspace quality under Gaussian sketching. For any matrix $A$ and any integer $k\ge0$, we derive an explicit, closed-form expression for the cumulative distribution function of the largest principal angle between the $k$-dominant singular subspace of $A$ and the approximate RRF subspace, expressing it in terms of a hypergeometric function. We obtain definitive probabilistic guarantees for RRFs that are strictly stronger than those obtained previously.

math.NA↗
arXiv2026

Rational Neural Networks have Expressivity Advantages

We study neural networks with trainable low-degree rational activation functions and show that they are more expressive and parameter-efficient than modern piecewise-linear and smooth activations such as ELU, LeakyReLU, LogSigmoid, PReLU, ReLU, SELU, CELU, Sigmoid, SiLU, Mish, Softplus, Tanh, Softmin, Softmax, and LogSoftmax. For an error target of $\varepsilon>0$, we establish approximation-theoretic separations: Any network built from standard fixed activations can be uniformly approximated on compact domains by a rational-activation network with only $\mathrm{poly}(\log\log(1/\varepsilon))$ overhead in size, while the converse provably requires $\Omega(\log(1/\varepsilon))$ parameters in the worst case. This exponential gap persists at the level of full networks and extends to gated activations and transformer-style nonlinearities. In practice, rational activations integrate seamlessly into standard architectures and training pipelines, allowing rationals to match or outperform fixed activations under identical architectures and optimizers.

cs.LG↗
arXiv2026

Linear Systems and Eigenvalue Problems: Open Questions from a Simons Workshop

This document presents a series of open questions arising in matrix computations, i.e., the numerical solution of linear algebra problems. It is a result of working groups at the workshop Linear Systems and Eigenvalue Problems, which was organized at the Simons Institute for the Theory of Computing program on Complexity and Linear Algebra in Fall 2025. The complexity and numerical solution of linear algebra problems is a crosscutting area between theoretical computer science and numerical analysis. The value of the particular problem formulations here is that they were produced via discussions between researchers from both groups. The open questions are organized in five categories: iterative solvers for linear systems, eigenvalue computation, low-rank approximation, randomized sketching, and other areas including tensors, quantum systems, and matrix functions. (Updated to reflect the status of the open problems as of August 20, 2026.)

math.NA↗
arXiv2025

Convergence of Pivoted Cholesky Algorithm for Lipschitz Kernels

We investigate the continuous analogue of the Cholesky factorization, namely the pivoted Cholesky algorithm. Our analysis establishes quantitative convergence guarantees for kernels of minimal smoothness. We prove that for a symmetric positive definite Lipschitz continuous kernel $K:\Omega\times \Omega \rightarrow \mathbb{R}$ on a compact domain $\Omega\subset\mathbb{R}^d$, the residual of the Cholesky algorithm with any pivoting strategy is uniformly bounded above by a constant multiple of the fill distance of pivots. In particular, our result implies that under complete pivoting (where the maximum value of the diagonal of the residual is selected as the next pivot): \begin{equation*} \|R_n\|_{\infty} = O(n^{-1/d}), \end{equation*} where $R_n$ is the residual after $n$ Cholesky steps and $\|\cdot\|_\infty$ is the absolute maximum value of $R_n$. Moreover, if $K$ is differentiable in both variables with a Lipschitz derivative, our convergence rate improves to $O(n^{-2/d})$. Our result closes a gap between theory and practice as previous analyses required $C^2$-regularity of $K$ to establish convergence, whereas empirical evidence indicated robust performance even for non-differentiable kernels. We further detail how our convergence results propagate to downstream applications, including discrete analogues, Gaussian process regression, and the P-greedy interpolation method.

math.NA↗
arXiv2025

The Distributional Koopman Operator for Random Dynamical Systems

The Distributional Koopman Operator (DKO) is introduced as a way to perform Koopman analysis on random dynamical systems where only aggregate distribution data is available, thereby eliminating the need for particle tracking or detailed trajectory data. Our DKO generalizes the stochastic Koopman operator (SKO) to allow for observables of probability distributions, using the transfer operator to propagate these probability distributions forward in time. Like the SKO, the DKO is linear with semigroup properties, and we show that the dynamical mode decomposition (DMD) approximation can converge to the DKO in the large data limit. The DKO is particularly useful for random dynamical systems where trajectory information is unavailable.

math.DS↗
arXiv2025

A Hidden Variable Resultant Method for the Polynomial Multiparameter Eigenvalue Problem

We present a novel, global algorithm for solving polynomial multiparameter eigenvalue problems (PMEPs) by leveraging a hidden variable tensor Dixon resultant framework. Our method transforms a PMEP into one or more univariate polynomial eigenvalue problems, which are solved as generalized eigenvalue problems. Our general approach avoids the need for custom linearizations of PMEPs. We provide rigorous theoretical guarantees for generic PMEPs and give practical strategies for nongeneric systems. Benchmarking on applications from aeroelastic flutter and leaky wave propagation confirms that our algorithm attains high accuracy and robustness while being broadly applicable to many PMEPs.

math.NA↗
arXiv2024

Extending Mercer's expansion to indefinite and asymmetric kernels

Mercer's expansion and Mercer's theorem are cornerstone results in kernel theory. While the classical Mercer's theorem only considers continuous symmetric positive definite kernels, analogous expansions are effective in practice for indefinite and asymmetric kernels. In this paper we extend Mercer's expansion to continuous kernels, providing a rigorous theoretical underpinning for indefinite and asymmetric kernels. We begin by demonstrating that Mercer's expansion may not be pointwise convergent for continuous indefinite kernels, before proving that the expansion of continuous kernels with bounded variation uniformly in each variable separably converges pointwise almost everywhere, almost uniformly, and unconditionally almost everywhere. We also describe an algorithm for computing Mercer's expansion for general kernels and give new decay bounds on its terms.

math.NA↗
arXiv2024

Numerical Instability of Algebraic Rootfinding Methods

We demonstrate that the most popular variants of all common algebraic multidimensional rootfinding algorithms are unstable by analyzing the conditioning of subproblems that are constructed at intermediate steps. In particular, we give multidimensional polynomial systems for which the conditioning of a subproblem can be worse than the conditioning of the original problem by a factor that grows exponentially with the number of variables.

math.NA↗
arXiv2024

How to reveal the rank of a matrix?

We study algorithms called rank-revealers that reveal a matrix's rank structure. Such algorithms form a fundamental component in matrix compression, singular value estimation, and column subset selection problems. While column-pivoted QR has been widely adopted due to its practicality, it is not always a rank-revealer. Conversely, Gaussian elimination (GE) with a pivoting strategy known as global maximum volume pivoting is guaranteed to estimate a matrix's singular values but its exponential complexity limits its interest to theory. We show that the concept of local maximum volume pivoting is a crucial and practical pivoting strategy for rank-revealers based on GE and QR. In particular, we prove that it is both necessary and sufficient; highlighting that all local solutions are nearly as good as the global one. This insight elevates Gu and Eisenstat's rank-revealing QR as an archetypal rank-revealer, and we implement a version that is observed to be at most $2\times$ more computationally expensive than CPQR. We unify the landscape of rank-revealers by considering GE and QR together and prove that the success of any pivoting strategy can be assessed by benchmarking it against a local maximum volume pivot.

math.NA↗
arXiv2024

Operator learning without the adjoint

There is a mystery at the heart of operator learning: how can one recover a non-self-adjoint operator from data without probing the adjoint? Current practical approaches suggest that one can accurately recover an operator while only using data generated by the forward action of the operator without access to the adjoint. However, naively, it seems essential to sample the action of the adjoint. In this paper, we partially explain this mystery by proving that without querying the adjoint, one can approximate a family of non-self-adjoint infinite-dimensional compact operators via projection onto a Fourier basis. We then apply the result to recovering Green's functions of elliptic partial differential operators and derive an adjoint-free sample complexity bound. While existing theory justifies low sample complexity in operator learning, ours is the first adjoint-free analysis that attempts to close the gap between theory and practice.

math.NA↗
arXiv2023

Operator learning for hyperbolic partial differential equations

We construct the first rigorously justified probabilistic algorithm for recovering the solution operator of a hyperbolic partial differential equation (PDE) in two variables from input-output training pairs. The primary challenge of recovering the solution operator of hyperbolic PDEs is the presence of characteristics, along which the associated Green's function is discontinuous. Therefore, a central component of our algorithm is a rank detection scheme that identifies the approximate location of the characteristics. By combining the randomized singular value decomposition with an adaptive hierarchical partition of the domain, we construct an approximant to the solution operator using $O(\Psi_\epsilon^{-1}\epsilon^{-7}\log(\Xi_\epsilon^{-1}\epsilon^{-1}))$ input-output pairs with relative error $O(\Xi_\epsilon^{-1}\epsilon)$ in the operator norm as $\epsilon\to0$, with high probability. Here, $\Psi_\epsilon$ represents the existence of degenerate singular values of the solution operator, and $\Xi_\epsilon$ measures the quality of the training data. Our assumptions on the regularity of the coefficients of the hyperbolic PDE are relatively weak given that hyperbolic PDEs do not have the ``instantaneous smoothing effect'' of elliptic and parabolic PDEs, and our recovery rate improves as the regularity of the coefficients increases.

math.NA↗
arXiv2023

A Mathematical Guide to Operator Learning

Operator learning aims to discover properties of an underlying dynamical system or partial differential equation (PDE) from data. Here, we present a step-by-step guide to operator learning. We explain the types of problems and PDEs amenable to operator learning, discuss various neural network architectures, and explain how to employ numerical PDE solvers effectively. We also give advice on how to create and manage training data and conduct optimization. We offer intuition behind the various neural network architectures employed in operator learning by motivating them from the point-of-view of numerical linear algebra.

math.NA↗