Retraction-Free Nonsmooth Optimization on the Stiefel Manifold: Subgradient and Proximal Landing Methods

Abstract

We develop three retraction-free Landing methods for nonsmooth composite optimization over the Stiefel manifold, combining an (approximately) tangential direction with an infeasibility-reducing normal direction. The subgradient Landing method, with normal stepsizes parameterized by $α\in[0,1]$, has best-iterate complexity bounds $\mathcal{O}(ε^{-4})$ for surrogate stationarity and $\mathcal{O}(ε^{-2/(2-α)})$ for feasibility. For $α=0$, it converges locally $R$-linearly under local weak sharpness and mild assumptions. The proximal gradient Landing method computes a tangential direction by solving a proximal subproblem and combines it with the same normal direction. Moreover, the inexact proximal gradient Landing method allows the proximal subproblem to be solved approximately and applies an explicit correction to control violations of the tangency condition. Both proximal methods have outer-iteration complexity bounds $\mathcal{O}(ε^{-2})$ for stationarity and $\mathcal{O}(ε^{-1})$ for feasibility, and every accumulation point is first-order stationary. Under an additional asymptotic Bregman-error condition and the Kurdyka-Lojasiewicz property of a suitable auxiliary function, the primal sequence of the inexact method converges to a first-order stationary point. Numerical experiments illustrate the effectiveness of all three methods. On the tested sparse principal component analysis instances, the inexact method has favorable running times, and both proximal methods maintain small feasibility residuals.