Single-loop $\mathcal{O}(\epsilon^{-3})$ stochastic smoothing algorithms for nonsmooth Riemannian optimization

Abstract

In this paper, we develop two Riemannian stochastic smoothing algorithms for nonsmooth optimization problems on Riemannian manifolds, addressing distinct forms of the nonsmooth term \( h \). Both methods combine dynamic smoothing with a momentum-based variance reduction scheme in a fully online manner. When \( h \) is Lipschitz continuous, we propose an stochastic algorithm under adaptive parameter that achieves the optimal iteration complexity of \( \mathcal{O}(\epsilon^{-3}) \), improving upon the best-known rates for exist algorithms. When \( h \) is the indicator function of a convex set, we design a new algorithm using truncated momentum, and under a mild error bound condition with parameter \( \theta \geq 1 \), we establish a complexity of \( \tilde{\mathcal{O}}(\epsilon^{-\max\{\theta+2, 2\theta\}}) \), in line with the best-known results in the Euclidean setting. Both algorithms feature a single-loop design with low per-iteration cost and require only \( \mathcal{O}(1) \) samples per iteration, ensuring that sample and iteration complexities coincide. Our framework encompasses a broad class of problems and recovers or matches optimal complexity guarantees in several important settings, including smooth stochastic Riemannian optimization, composite problems in Euclidean space, and constrained optimization via indicator functions.