Tensor Methods for Finding Approximate Stationary Points of Convex Functions

Abstract

In this paper we consider the problem of finding $ε$-approximate stationary points of convex functions that are $p$-times differentiable with $ν$-Hölder continuous $p$th derivatives. We present tensor methods with and without acceleration. Specifically, we show that the non-accelerated schemes take at most $\mathcal{O}\left(ε^{-1/(p+ν-1)}\right)$ iterations to reduce the norm of the gradient of the objective below a given $ε\in (0,1)$. For accelerated tensor schemes we establish improved complexity bounds of $\mathcal{O}\left(ε^{-(p+ν)/[(p+ν-1)(p+ν+1)]}\right)$ and $\mathcal{O}\left(|\log(ε)|ε^{-1/(p+ν)}\right)$, when the Hölder parameter $ν\in [0,1]$ is known. For the case in which $ν$ is unknown, we obtain a bound of $\mathcal{O}\left(ε^{-(p+1)/[(p+ν-1)(p+2)]}\right)$ for a universal accelerated scheme. Finally, we also obtain a lower complexity bound of $\mathcal{O}\left(ε^{-2/[3(p+ν)-2]}\right)$ for finding $ε$-approximate stationary points using $p$-order tensor methods.