Synthesis of the Transient Behaviour of Optimization Algorithms for Strongly Convex Objective Functions

Abstract

We study the transient behavior of first-order optimization methods for unconstrained minimization of smooth and strongly convex objective functions. These methods are modelled as linear systems over finite-time intervals embedded in feedback with the gradient nonlinearity. By characterizing the function class through discrete-time Zames-Falb integral quadratic constraints, we employ a robust $\mathcal{H}_2$ control framework to obtain convex semidefinite programs certifying tight worst-case bounds on the terminal distance to the optimizer. Building on the same formulation, we derive a convex synthesis procedure for a transient-optimal method that achieves the optimal finite-horizon bounds and guarantees error reduction over any prescribed number of steps. The framework naturally extends to delayed gradient measurements, enabling both transient performance analysis and synthesis of optimal algorithms under feedback delays.