arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.10369

Sharp Initialization Guarantees for the Heavy-Ball Method

Kang Liu·Quanen Wu·Zhan Sun·Mengxiao Chen·Suyan Li·Bohao Qu
View original source ↗Read source document ↗Export metadata ↓

Abstract

The heavy-ball method can accelerate convergence near the minimizer, but it may also enter persistent oscillations on general strongly convex objectives. How should it be initialized to reliably realize this local advantage? We study the initialization neighborhood in which convergence can be guaranteed using only curvature and regularity bounds. This neighborhood shrinks as momentum approaches one; for Lipschitz-continuous Hessians, its radius is of the same order as damping. Matching upper and lower bounds show that this initialization requirement cannot be relaxed uniformly over the function class. We then derive a startup rule based on the current gradient: a finite gradient-descent warm-up enters the guaranteed region before switching to heavy-ball. With parameters optimal for strongly convex quadratics, the subsequent iterates satisfy an accelerated local asymptotic rate bound. Code is available at https://anonymous.4open.science/r/HeavyBall-811D/.

Explore related subjects

math.OC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Kang Liu, Quanen Wu, Zhan Sun, Mengxiao Chen, Suyan Li, Bohao Qu. 2026-10-07. Sharp Initialization Guarantees for the Heavy-Ball Method. https://arxiv.org/abs/2610.10369

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Perturbed Iterate SGD for Lipschitz Continuous Loss Functions with Numerical Error and Adaptive Step Sizes

Motivated by neural network training in finite-precision arithmetic environments, this work studies the convergence of perturbed iterate SGD using adaptive step sizes in an environment with numerical error. Considering a general stochastic Lipschitz continuous loss function, an asymptotic convergence result to a Clarke stationary point is proven as well as the non-asymptotic convergence to an approximate stationary point in expectation. It is assumed that only an approximation of the loss function's stochastic gradient can be computed, in addition to error in computing the SGD step itself.

math.OC↗
arXiv2026

Controllability Allocation Scores for Targeted Network Intervention

We introduce the controllability allocation score (CAS), a framework for determining how intervention intensity should be distributed among prescribed candidate input directions with respect to designated target variables, together with the target controllability score (TCS) as its nodewise specialization. We establish existence of the CASs and develop a general uniqueness theory based on restricted injectivity of the allocation-to-Gramian map, including generic uniqueness with respect to the time horizon. We show that restricting attention to target variables can fundamentally alter the optimal intervention allocation compared with the standard full-state setting. To enable scalability, we develop a general surrogate-Gramian framework and derive objective-performance guarantees from relative Gramian errors without requiring uniqueness or closeness of the CASs. For the TCS specialization, we further construct a target-only reduced virtual system and derive explicit bounds showing how the approximation error depends on the coupling between target and non-target nodes and on the dynamical growth rate. Experiments on human brain networks show that the reduced formulation accurately approximates the TCS at short horizons, whereas the two controllability criteria exhibit markedly different approximation accuracy at long horizons.

math.OC↗
arXiv2026

Solving the Offline and Online Min-Max Problem of Non-smooth Submodular-Concave Functions: A Zeroth-Order Approach

We consider max-min and min-max problems with objective functions that are possibly non-smooth, submodular with respect to the minimiser and concave with respect to the maximiser. We investigate the performance of a zeroth-order method applied to this problem. The method is based on the subgradient of the Lovász extension of the objective function with respect to the minimiser and based on Gaussian smoothing to estimate the smoothed function gradient with respect to the maximiser. In expectation sense, we prove the convergence of the algorithm to an $ε$-saddle point in the offline case. Moreover, we show that, in the expectation sense, in the online setting, the algorithm achieves $O(\sqrt{N(1+\bar{P}_N)})$ online duality gap, where $N$ is the number of iterations and $\bar{P}_N$ is the path length of the sequence of optimal decisions. The complexity analysis and hyperparameter selection are presented for all the cases. The theoretical results are illustrated via numerical examples.

math.OC↗