arXiv subjects

Weiming Ou

Publications and source records attributed to Weiming Ou.

3 recordsLinked to original sources

arXiv2026

Tight Sampling Complexity with stochastic gradient oracles in Fixed Dimensions

We investigate the stochastic-gradient query complexity of sampling smooth strongly log-concave distributions in any fixed Euclidean dimension. The potential is $μ$-strongly convex and $L$-smooth, with an unknown mode in the ball of radius $μ^{-1/2}$ about the origin. We have access to unbiased stochastic oracles with the variance at most $σ^2$. For every $σ^2\ge0$ and total variation (TV) accuracy $0<\varepsilon\le1/10$, we prove that the tight complexity of sampling a distribution within $ε$-TV distance from the target distribution is \[ N^\star_{\text{TV}}=Θ\!\left(\log(1+κ)+ \frac{σ^2}{με}\right), \] where $κ:=\frac Lμ$ is the condition number. Note that this complexity bound is simultaneously tight for the condition number $κ$ and accuracy $ε$. Besides, our tight complexity bound is adaptive to noiseless setting $σ=0$, which is $ N^\star_{\text{TV}}=Θ\!\left(\log(1+κ)\right)$.

math.ST↗

arXiv2026

Heavy-Tailed First-Order Optimization for Polyak-Łojasiewicz Condition: High-Dimensional Minimax Bounds, High-Probability Guarantee, and Fixed-Dimensional Improvements

We study smooth Polyak--Łojasiewicz (PL) optimization with conditionally unbiased stochastic gradients satisfying \[ \mathbb E\!\left[ \|G_t-\nabla f(x_t)\|^α\mid\mathcal F_{t-1} \right]\le σ^α, \qquad 1<α\le2. \] When the dimension may depend on the oracle budget, we prove the noise-adaptive lower bound \[ T_ε= Ω_α\!\left[ κ\log\frac{Δ_0}ε + κ\left( \frac{σ^2}{με} \right)^{\fracα{2(α-1)}} \right], \] which recovers the noiseless PL lower bound when $σ=0$. Under the appropriate mirror-PL condition, we give a centered-clipped mirror-descent method attaining the matching high-probability upper bound up to logarithmic factors, without bounded-domain, bounded-gradient, or sub-Gaussian assumptions. We further characterize the stochastic complexity in prescribed fixed dimensions. For $d=1,2,3$, the optimal stochastic term is \[ \widetildeΘ_α\!\left[ \left( \frac{σ^2}{με} \right)^{\fracα{2(α-1)}} \right]. \] For every fixed $d>3$, the same characterization holds whenever \[ \fracα{α-1}\ge d-1. \] In the complementary regime, we provide an upper bound with an additional surface-entropy factor and explicitly identify the remaining gap.

math.OC↗

arXiv2026

Understanding Dynamics of Adam in Zero-Sum Games: An ODE Approach

The remarkable success of the Adam in training neural networks has naturally led to the widespread use of its descent-ascent counterpart, Adam-DA, for solving zero-sum games. Despite its popularity in practice, a rigorous theoretical understanding of Adam-DA still lags behind. In this paper, we derive ordinary differential equations (ODEs) that serve as continuous-time limits of the Adam-DA. These ODEs closely approximate the discrete-time dynamics of Adam-DA, providing a tractable analytical framework for understanding its behavior in zero-sum games. Using this ODE approach, we investigate two fundamental aspects of Adam-DA: local convergence and implicit gradient regularization. Our analysis reveals that the roles of the first- and second-order momentum parameters in zero-sum games are exactly the opposite of their well-documented effects in minimization problems. We validate these predictions through GAN experiments across multiple architectures and datasets, demonstrating the practical implications of this reversed momentum effect.

cs.LG↗