arXiv subjects

Shichen Liao

Publications and source records attributed to Shichen Liao.

2 recordsLinked to original sources

arXiv2026

Robust Accelerated Adaptive Search: High-Probability Complexity Bounds under Bounded-Moment Stochastic Oracles

We study unconstrained smooth convex optimization under stochastic first- and zeroth-order oracles subject only to finite-moment bounds, naturally admitting persistent bias and heavy-tailed noise. In this hostile environment, integrating momentum into \emph{adaptive step search} to secure acceleration poses an inherent structural challenge, because momentum propagates oracle errors across iterations, inevitably undermining the stabilizing effect of local search. To address this difficulty, we propose \texttt{RAAS}, a robust accelerated adaptive search method with tunable momentum intervention. Theoretically, we develop a general high-probability framework for adaptive search methods under stochastic oracle feedback, and instantiate it through the strongly convex and general convex analyses of \texttt{RAAS}. This yields high-probability stopping-time complexity bounds for reaching the attainable precision neighborhood. The resulting guarantees also clarify how the algorithmic parameters trade off early-stage acceleration against late-stage stability, and motivate a simple switching heuristic that performs well empirically.

math.OC↗

arXiv2024

Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-error

Establishing robust policies is essential to counter attacks or disturbances affecting deep reinforcement learning (DRL) agents. Recent studies explore state-adversarial robustness and suggest the potential lack of an optimal robust policy (ORP), posing challenges in setting strict robustness constraints. This work further investigates ORP: At first, we introduce a consistency assumption of policy (CAP) stating that optimal actions in the Markov decision process remain consistent with minor perturbations, supported by empirical and theoretical evidence. Building upon CAP, we crucially prove the existence of a deterministic and stationary ORP that aligns with the Bellman optimal policy. Furthermore, we illustrate the necessity of $L^{\infty}$-norm when minimizing Bellman error to attain ORP. This finding clarifies the vulnerability of prior DRL algorithms that target the Bellman optimal policy with $L^{1}$-norm and motivates us to train a Consistent Adversarial Robust Deep Q-Network (CAR-DQN) by minimizing a surrogate of Bellman Infinity-error. The top-tier performance of CAR-DQN across various benchmarks validates its practical effectiveness and reinforces the soundness of our theoretical analysis.

cs.LG↗