arXiv subjects

Yanyi Li

Publications and source records attributed to Yanyi Li.

4 recordsLinked to original sources

arXiv2026

Near-Optimal Deterministic Exact-Value Complexity for Smooth Convex Optimization

We study the deterministic oracle complexity of smooth convex optimization when the algorithm receives only exact function values. The objective is a globally $β$-smooth convex function, all queries and the final output are restricted to the Euclidean ball of radius $R$, and the unique minimizer lies in the ball of radius $R/2$. We establish an upper bound of $O(d\sqrt{βR^2/ε})$ using coordinate finite differences together with an error-robust accelerated projected method. Our main contribution is a matching lower bound, up to the high-accuracy saturation of the construction: any deterministic adaptive value-oracle algorithm requires $Ω\!\left(d\min\{\sqrt{βR^2/ε},(d/\log(ed))^{1/3}\}\right)$ queries. Consequently, the minimax oracle complexity is $Θ(d\sqrt{βR^2/ε})$ throughout the moderate-accuracy regime $βR^2(\log(ed)/d)^{2/3}\leqε\leq cβR^2$ for a universal constant $c>0$. The lower bound must account for the fact that a single exact real value can encode arbitrarily much information. To overcome this difficulty, we construct a single fixed smooth convex hard instance using a Moreau-smoothed biased max chain, an exact prefix-shielding mechanism, and batched delayed rotations. These techniques preserve consistency with the full adaptive transcript and establish the optimality of the square-root complexity branch for deterministic bounded-query algorithms.

math.OC↗

arXiv2026

Near-Optimal Exact-Value Zeroth-Order Complexity for Smooth Strongly Convex Optimization

We study deterministic adaptive optimization of globally $β$-smooth, $μ$-strongly convex functions using exact scalar function values. Queries and outputs lie in $B_2^d(R)$, and the minimizer lies in $B_2^d(R/2)$. Set $κ=β/μ$, $Q=βR^2/ε$, and $D_d=(d/\log(ed))^{1/3}$. For sufficiently large $d$ and $0<ε\le c_εβR^2$, the minimax value complexity $N_ε$ satisfies \[ \begin{aligned} N_ε&\ge c d\min\{\sqrt Q,\sqrtκ,D_d\},\\ N_ε&\le C d\min\left\{ \sqrt Q,\sqrtκ[1+\log_+(Q/κ)] \right\}, \end{aligned} \] where $c,C,c_ε>0$ are universal constants and $\log_+(t)=\max\{0,\log t\}$. The lower bound uses an exactly shielded smooth chain and batched delayed rotations; the upper bound combines finite differences, acceleration, and restart. When $\min\{Q,κ\}\le D_d^2$, these bounds match up to constants in the accuracy-dominated regime $Q\leκ$ and at constant relative accuracy $ε=Θ(μR^2)$. For arbitrarily higher accuracy in the range $κ\le D_d^2$, the bounds differ by at most $1+\log(μR^2/ε)$; the optimal accuracy dependence remains unresolved in general.

math.OC↗

arXiv2026

PRAC: Principal-Random Subspace for LLM Activation Compression and Memory-Efficient Training

Activations have become the primary memory bottleneck in large-batch LLM training. However, existing compression methods fail to exploit the spectral structure of activations, resulting in slow convergence or limited compression. To address this, we bridge the relationship between the algorithm's fast convergence and the requirements for subspace projection, and show that an effective compression should yield an unbiased estimate of the original activation with low variance. We propose Principal-Random Subspace for LLM Activation Compression (PRAC), which novelly decomposes activations into two components: a principal subspace captured via SVD to retain dominant information, and a random subspace sampled from the orthogonal complement to approximate the tail. By introducing a precise scaling factor, we prove that PRAC yields an unbiased gradient estimator with minimum variance under certain conditions. Extensive experiments on pre-training and fine-tuning tasks demonstrate that PRAC achieves up to 36% total memory reduction with negligible performance degradation and minimal computational cost.

cs.LG↗

arXiv2023

MS-UNet-v2: Adaptive Denoising Method and Training Strategy for Medical Image Segmentation with Small Training Data

Models based on U-like structures have improved the performance of medical image segmentation. However, the single-layer decoder structure of U-Net is too "thin" to exploit enough information, resulting in large semantic differences between the encoder and decoder parts. Things get worse if the number of training sets of data is not sufficiently large, which is common in medical image processing tasks where annotated data are more difficult to obtain than other tasks. Based on this observation, we propose a novel U-Net model named MS-UNet for the medical image segmentation task in this study. Instead of the single-layer U-Net decoder structure used in Swin-UNet and TransUnet, we specifically design a multi-scale nested decoder based on the Swin Transformer for U-Net. The proposed multi-scale nested decoder structure allows the feature mapping between the decoder and encoder to be semantically closer, thus enabling the network to learn more detailed features. In addition, we propose a novel edge loss and a plug-and-play fine-tuning Denoising module, which not only effectively improves the segmentation performance of MS-UNet, but could also be applied to other models individually. Experimental results show that MS-UNet could effectively improve the network performance with more efficient feature learning capability and exhibit more advanced performance, especially in the extreme case with a small amount of training data, and the proposed Edge loss and Denoising module could significantly enhance the segmentation performance of MS-UNet.

eess.IV↗