arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2503.16312

Near-Linear Runtime for a Classical Matrix Preconditioning Algorithm

Xufeng Cai·Jason M. Altschuler·Jelena Diakonikolas
View original source ↗Read source document ↗Export metadata ↓

Abstract

In 1960, Osborne proposed a simple iterative algorithm for matrix balancing with outstanding numerical performance. Today, it is the default preconditioning procedure before eigenvalue computation and other linear algebra subroutines in mainstream software packages such as Python, Julia, MATLAB, EISPACK, LAPACK, and more. Despite its widespread usage, Osborne's algorithm has long resisted theoretical guarantees for its runtime: the first polynomial-time guarantees were obtained only in the past decade, and recent near-linear runtimes remain confined to variants of Osborne's algorithm with important differences that make them simpler to analyze but empirically slower. In this paper, we address this longstanding gap between theory and practice by proving that Osborne's original algorithm -- the de facto preconditioner in practice -- in fact has a near-linear runtime. This runtime guarantee (1) is optimal in the input size up to at most a single logarithm, (2) is the first runtime for Osborne's algorithm that does not dominate the runtime of downstream tasks like eigenvalue computation, and (3) improves upon the theoretical runtimes for all other variants of Osborne's algorithm.

Explore related subjects

math.OCcs.DScs.NAmath.NA

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Xufeng Cai, Jason M. Altschuler, Jelena Diakonikolas. 2025-03-20. Near-Linear Runtime for a Classical Matrix Preconditioning Algorithm. https://arxiv.org/abs/2503.16312

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Stochastic Optimization Algorithms for Problems with Controllable Biased Oracles

Motivated by emerging applications in machine learning, we consider an optimization problem in a general setting in which the gradient of the objective function is available via a biased stochastic oracle. We assume a bias-control parameter can reduce the bias magnitude; however, a lower bias requires more computation/samples. For instance, in two applications on stochastic composition optimization and policy optimization for infinite-horizon Markov decision processes, we show that the bias follows a power law and exponential decay, respectively, as functions of their corresponding bias control parameters. For problems with such gradient oracles, the paper proposes stochastic algorithms that adjust the bias-control parameter throughout the iterations. We analyze the nonasymptotic performance of the proposed algorithms in the nonconvex regime and establish their sample or bias-control computation complexities to obtain a stationary point in expectation or with high probability. Finally, we numerically evaluate the performance of the proposed algorithms over three applications.

math.OC↗
arXiv2026

How long is long enough? Finite-horizon approximation of energy storage scheduling problems

Energy storage scheduling problems, where a storage system is operated to maximize its profit in response to a price signal, are naturally formulated as infinite-horizon optimization problems, since storage systems operate continuously, without a foreseen end to their operation. Such problems can be solved to optimality with a rolling-horizon approach, provided that the planning horizon over which the problem is solved is long enough. Such a horizon is termed a forecast horizon. Despite its importance, the planning horizon is usually chosen arbitrarily for such applications. We introduce an easy-to-check condition that confirms whether a planning horizon is a forecast horizon, and which can be used to derive a bound on suboptimality when it is not the case. In practice, this condition enables practitioners to evaluate whether the planning horizon chosen is sufficient, thereby providing, for the first time, a practical means of evaluating and selecting planning horizons for energy storage scheduling problems. We also derive a lower bound on the minimum forecast horizon. Building on the theoretical results, we develop an algorithm to determine the minimum forecast horizon. It enables the identification, a posteriori, of the shortest planning horizon that guarantees optimal rolling-horizon decisions while avoiding unnecessary forecasting effort and computational cost. Numerical experiments illustrate the practical use of the proposed framework and investigate how storage system characteristics and electricity price patterns influence the minimum forecast horizon.

math.OC↗
arXiv2026

Inverse Problems Over Probability Measure Space

Define a forward problem as $ρ_y = G_\#ρ_x$, where the probability distribution $ρ_x$ is mapped to another distribution $ρ_y$ using the forward operator $G$. In this work, we investigate the corresponding inverse problem: Given $ρ_y$, how to find $ρ_x$? Depending on whether $ G$ is overdetermined or underdetermined, the solution can have drastically different behavior. In the overdetermined case, we formulate a variational problem $\min_{ρ_x} D( G_\#ρ_x, ρ_y)$, and find that different choices of the metric $ D$ significantly affect the quality of the reconstruction. When $ D$ is set to be the Wasserstein distance, the reconstruction is the marginal distribution, while setting $ D$ to be a $ϕ$-divergence reconstructs the conditional distribution. In the underdetermined case, we formulate the constrained optimization $\min_{\{ G_\#ρ_x=ρ_y\}} E[ρ_x]$. The choice of $ E$ also significantly impacts the construction: setting $ E$ to be the entropy gives us the piecewise constant reconstruction, while setting $ E$ to be the second moment, we recover the classical least-norm solution. We also examine the formulation with regularization: $\min_{ρ_x} D( G_\#ρ_x, ρ_y) + α\mathsf R[ρ_x]$, and find that the entropy-entropy pair leads to a regularized solution that is defined in a piecewise manner, whereas the $W_2$-$W_2$ pair leads to a least-norm solution where $W_2$ is the 2-Wasserstein metric.

math.OC↗