Tensor Methods for Minimizing Convex Functions with Hölder Continuous Higher-Order Derivatives

Abstract

In this paper we study $p$-order methods for unconstrained minimization of convex functions that are $p$-times differentiable ($p\geq 2$) with $ν$-Hölder continuous $p$th derivatives. We propose tensor schemes with and without acceleration. For the schemes without acceleration, we establish iteration complexity bounds of $\mathcal{O}\left(ε^{-1/(p+ν-1)}\right)$ for reducing the functional residual below a given $ε\in (0,1)$. Assuming that $ν$ is known, we obtain an improved complexity bound of $\mathcal{O}\left(ε^{-1/(p+ν)}\right)$ for the corresponding accelerated scheme. For the case in which $ν$ is unknown, we present a universal accelerated tensor scheme with iteration complexity of $\mathcal{O}\left(ε^{-p/[(p+1)(p+ν-1)]}\right)$. A lower complexity bound of $\mathcal{O}\left(ε^{-2/[3(p+ν)-2]}\right)$ is also obtained for this problem class.