Robust Accelerated Adaptive Search: High-Probability Complexity Bounds under Bounded-Moment Stochastic Oracles

Abstract

We study unconstrained smooth convex optimization under stochastic first- and zeroth-order oracles subject only to finite-moment bounds, naturally admitting persistent bias and heavy-tailed noise. In this hostile environment, integrating momentum into \emph{adaptive step search} to secure acceleration poses an inherent structural challenge, because momentum propagates oracle errors across iterations, inevitably undermining the stabilizing effect of local search. To address this difficulty, we propose \texttt{RAAS}, a robust accelerated adaptive search method with tunable momentum intervention. Theoretically, we develop a general high-probability framework for adaptive search methods under stochastic oracle feedback, and instantiate it through the strongly convex and general convex analyses of \texttt{RAAS}. This yields high-probability stopping-time complexity bounds for reaching the attainable precision neighborhood. The resulting guarantees also clarify how the algorithmic parameters trade off early-stage acceleration against late-stage stability, and motivate a simple switching heuristic that performs well empirically.