arXiv subjects

Wang Luo

Publications and source records attributed to Wang Luo.

22 records · Page 2Linked to original sources

arXiv2025

Towards Optimal Adversarial Robust Reinforcement Learning with Infinity Measurement Error

Ensuring the robustness of deep reinforcement learning (DRL) agents against adversarial attacks is critical for their trustworthy deployment. Recent research highlights the challenges of achieving state-adversarial robustness and suggests that an optimal robust policy (ORP) does not always exist, complicating the enforcement of strict robustness constraints. In this paper, we further explore the concept of ORP. We first introduce the Intrinsic State-adversarial Markov Decision Process (ISA-MDP), a novel formulation where adversaries cannot fundamentally alter the intrinsic nature of state observations. ISA-MDP, supported by empirical and theoretical evidence, universally characterizes decision-making under state-adversarial paradigms. We rigorously prove that within ISA-MDP, a deterministic and stationary ORP exists, aligning with the Bellman optimal policy. Our findings theoretically reveal that improving DRL robustness does not necessarily compromise performance in natural environments. Furthermore, we demonstrate the necessity of infinity measurement error (IME) in both $Q$-function and probability spaces to achieve ORP, unveiling vulnerabilities of previous DRL algorithms that rely on $1$-measurement errors. Motivated by these insights, we develop the Consistent Adversarial Robust Reinforcement Learning (CAR-RL) framework, which optimizes surrogates of IME. We apply CAR-RL to both value-based and policy-based DRL algorithms, achieving superior performance and validating our theoretical analysis.

cs.LG↗

arXiv2024

Noncollinear ferroelectric and screw-type antiferroelectric phases in a metal-free hybrid molecular crystal

Noncollinear dipole textures greatly extend the scientific merits and application perspective of ferroic materials. In fact, noncollinear spin textures have been well recognized as one of the core issues of condensed matter, e.g. cycloidal/conical magnets with multiferroicity and magnetic skyrmions with topological properties. However, the counterparts in electrical polarized materials are less studied and thus urgently needed, since electric dipoles are usually aligned collinearly in most ferroelectrics/antiferroelectrics. Molecular crystals with electric dipoles provide a rich ore to explore the noncollinear polarity. Here we report an organic salt (H2Dabco)BrClO4 (H2Dabco = N,N'-1,4-diazabicyclo[2.2.2]octonium) that shows a transition between the ferroelectric and antiferroelectric phases. Based on experimental characterizations and ab initio calculations, it is found that its electric dipoles present nontrivial noncollinear textures with $60^\circ$-twisting angle between the neighbours. Then the ferroelectric-antiferroelectric transition can be understood as the coding of twisting angle sequence. Our study reveals the unique science of noncollinear electric polarity.

cond-mat.mtrl-sci↗

arXiv2024

Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-error

Establishing robust policies is essential to counter attacks or disturbances affecting deep reinforcement learning (DRL) agents. Recent studies explore state-adversarial robustness and suggest the potential lack of an optimal robust policy (ORP), posing challenges in setting strict robustness constraints. This work further investigates ORP: At first, we introduce a consistency assumption of policy (CAP) stating that optimal actions in the Markov decision process remain consistent with minor perturbations, supported by empirical and theoretical evidence. Building upon CAP, we crucially prove the existence of a deterministic and stationary ORP that aligns with the Bellman optimal policy. Furthermore, we illustrate the necessity of $L^{\infty}$-norm when minimizing Bellman error to attain ORP. This finding clarifies the vulnerability of prior DRL algorithms that target the Bellman optimal policy with $L^{1}$-norm and motivates us to train a Consistent Adversarial Robust Deep Q-Network (CAR-DQN) by minimizing a surrogate of Bellman Infinity-error. The top-tier performance of CAR-DQN across various benchmarks validates its practical effectiveness and reinforces the soundness of our theoretical analysis.

cs.LG↗

arXiv2021

Action Unit Memory Network for Weakly Supervised Temporal Action Localization

Weakly supervised temporal action localization aims to detect and localize actions in untrimmed videos with only video-level labels during training. However, without frame-level annotations, it is challenging to achieve localization completeness and relieve background interference. In this paper, we present an Action Unit Memory Network (AUMN) for weakly supervised temporal action localization, which can mitigate the above two challenges by learning an action unit memory bank. In the proposed AUMN, two attention modules are designed to update the memory bank adaptively and learn action units specific classifiers. Furthermore, three effective mechanisms (diversity, homogeneity and sparsity) are designed to guide the updating of the memory network. To the best of our knowledge, this is the first work to explicitly model the action units with a memory network. Extensive experimental results on two standard benchmarks (THUMOS14 and ActivityNet) demonstrate that our AUMN performs favorably against state-of-the-art methods. Specifically, the average mAP of IoU thresholds from 0.1 to 0.5 on the THUMOS14 dataset is significantly improved from 47.0% to 52.1%.

cs.CV↗