Large-Step Last-Iterate Convergence of Random-Reshuffling Stochastic Gradient Descent

Zijun Chen; Zihan Zhang

Home/Archive/arXiv

arXiv · 2610.19444

Large-Step Last-Iterate Convergence of Random-Reshuffling Stochastic Gradient Descent

Zijun Chen·Zihan Zhang

View original source ↗Read source document ↗Export metadata ↓

Abstract

Random reshuffling stochastic gradient descent (RR-SGD) is widely used for finite-sum optimization in machine learning. For convex $L$-smooth components, recent work establishes convergence guarantees for an averaged iterate with component stepsizes as large as order $1/L$. However, the final iterate is a natural output in practice, and existing last-iterate guarantees in this setting require stepsizes that scale inversely with the number of components $n$. We establish an expected objective-gap bound for the last iterate of standard RR-SGD on unconstrained problems with a constant component stepsize satisfying $0<η\lesssim 1/(L(1 + \log(nK)))$, where $K$ is the number of epochs. By retaining the sampling bias at random comparison points, we remove the explicit $1/n$ factor from the stepsize restriction. Combined with existing small-step bounds, our result matches the corresponding averaged-iterate convergence rate up to logarithmic factors.

Explore related subjects

math.OC cs.LG stat.ML

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Zijun Chen, Zihan Zhang. 2026-10-14. Large-Step Last-Iterate Convergence of Random-Reshuffling Stochastic Gradient Descent. https://arxiv.org/abs/2610.19444

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A Safe Approximation Based on Mixed-Integer Optimization for Non-Convex Distributional Robustness Governed by Univariate Indicator Functions

In this work, we present an algorithmically tractable safe approximation of distributionally robust optimization (DRO) problems that contain univariate indicator functions. The latter appear in different applications, but render the model nonlinear and nonconvex. The considered ambiguity sets can exploit moment information. Typically, reformulation approaches using duality theory need to make strong assumptions on the structure of the underlying constraints, such as convexity in the decisions or concavity in the uncertainty which cannot be assumed in our setting. We nevertheless present a semi-infinite minorant of the adversarial problem that admits a zero-gap dualization and is subsequently approximated by a discretized counterpart. Under mild assumptions, the latter provides a safe approximation that is a mixed-integer linear problem and can be solved by available standard software. Obtained solutions are guaranteed to be feasible for the original distributionally robust problem. We prove that the approximation of the adversarial problem converges to its true value for increasingly fine discretization, and that the values of the solutions of the safe approximations converge to the value of the true DRO problem under mild assumptions. In addition, the error bound yields an a posteriori enclosure of the true optimal value that can be evaluated in practice. On the practical side, the approach is made concrete for a challenging, fundamental task in material design, namely in particle separation. Computational results for a realistic setting show that the safe approximation meets the imposed quality requirement for every distribution in the ambiguity set, whereas the unprotected solution does not.

math.OC↗

arXiv2026

Geometric Certificates for Deterministic Global Linear Convergence of Overparameterized Projected Gradient Descent

We study projected gradient descent (ProjGD) for the rank-constrained problem $\min\{f(X): \operatorname{rank}(X)\le r\}$ when the unconstrained minimizer $X_*$ has rank $r_*\le r$. Our main contribution is a quantitative geometric certificate relating the normal component of $\nabla f(X)$ to the smallest nonzero singular value of $X$. Under a rank-$(r+r_*)$ restricted curvature condition with constants $0<μ \sqrt{r_*/r}(L-μ)/(2Lμ)$. With an additional rank-$2r$ restricted smoothness assumption, we establish deterministic global linear convergence for $\sqrt{r_*/r}(L-μ)/(2Lμ)<η<1/L$. The result accommodates arbitrary rank overparameterization $r\ge r_*$ and is independent of the singular-value condition number of $X_*$. As a consequence, we obtain an alternative proof of the sharp threshold $δ<(1+\sqrt{r_*/r})^{-1}$ for the absence of spurious local minima in the asymmetric setting, where $δ=(L-μ)/(L+μ)$. This threshold and matching asymmetric examples were established in recent work.

math.OC↗

arXiv2026

Congestion-Aware Charging Coordination for Electric Ride-Hailing Fleets under Stochastic Demand

In this study, we develop a dynamic charging scheduling method that integrates day-ahead charging planning and online reactive vehicle dispatch and charger assignment. The day-ahead planning model minimizes expected charging costs by considering anticipated waiting times and overnight energy replenishment costs. The online model then adapts this plan by maximizing the profit from vehicle dispatching and assigning vehicles to chargers through a reservation-based charging request scheduler to minimize the total charging-related operational time. The developed charging policy is tested on a Manhattan-like study area using synthetic data drawn from NYC yellow taxi data with a fleet size of 100 vehicles given the scenarios of 3000 and 4000 customers/day. The computational results show that our method outperforms different benchmark policies with up to +21.68% profit and +22.22% service rate for 4000 customers relative to the weakest benchmark. Sensitivity analyses are conducted for different system parameters, and managerial insights are discussed.

math.OC↗