arXiv subjects

Yang Yu

Publications and source records attributed to Yang Yu.

At least 181 records · Page 10Linked to original sources

arXiv2025

NeoRL-2: Near Real-World Benchmarks for Offline Reinforcement Learning with Extended Realistic Scenarios

Offline reinforcement learning (RL) aims to learn from historical data without requiring (costly) access to the environment. To facilitate offline RL research, we previously introduced NeoRL, which highlighted that datasets from real-world tasks are often conservative and limited. With years of experience applying offline RL to various domains, we have identified additional real-world challenges. These include extremely conservative data distributions produced by deployed control systems, delayed action effects caused by high-latency transitions, external factors arising from the uncontrollable variance of transitions, and global safety constraints that are difficult to evaluate during the decision-making process. These challenges are underrepresented in previous benchmarks but frequently occur in real-world tasks. To address this, we constructed the extended Near Real-World Offline RL Benchmark (NeoRL-2), which consists of 7 datasets from 7 simulated tasks along with their corresponding evaluation simulators. Benchmarking results from state-of-the-art offline RL approaches demonstrate that current methods often struggle to outperform the data-collection behavior policy, highlighting the need for more effective methods. We hope NeoRL-2 will accelerate the development of reinforcement learning algorithms for real-world applications. The benchmark project page is available at https://github.com/polixir/NeoRL2.

cs.LG↗

arXiv2025

VA-AR: Learning Velocity-Aware Action Representations with Mixture of Window Attention

Action recognition is a crucial task in artificial intelligence, with significant implications across various domains. We initially perform a comprehensive analysis of seven prominent action recognition methods across five widely-used datasets. This analysis reveals a critical, yet previously overlooked, observation: as the velocity of actions increases, the performance of these methods variably declines, undermining their robustness. This decline in performance poses significant challenges for their application in real-world scenarios. Building on these findings, we introduce the Velocity-Aware Action Recognition (VA-AR) framework to obtain robust action representations across different velocities. Our principal insight is that rapid actions (e.g., the giant circle backward in uneven bars or a smash in badminton) occur within short time intervals, necessitating smaller temporal attention windows to accurately capture intricate changes. Conversely, slower actions (e.g., drinking water or wiping face) require larger windows to effectively encompass the broader context. VA-AR employs a Mixture of Window Attention (MoWA) strategy, dynamically adjusting its attention window size based on the action's velocity. This adjustment enables VA-AR to obtain a velocity-aware representation, thereby enhancing the accuracy of action recognition. Extensive experiments confirm that VA-AR achieves state-of-the-art performance on the same five datasets, demonstrating VA-AR's effectiveness across a broad spectrum of action recognition scenarios.

cs.CV↗

arXiv2025

Q-Adapter: Customizing Pre-trained LLMs to New Preferences with Forgetting Mitigation

Large Language Models (LLMs), trained on a large amount of corpus, have demonstrated remarkable abilities. However, it may not be sufficient to directly apply open-source LLMs like Llama to certain real-world scenarios, since most of them are trained for \emph{general} purposes. Thus, the demands for customizing publicly available LLMs emerge, but are currently under-studied. In this work, we consider customizing pre-trained LLMs with new human preferences. Specifically, the LLM should not only meet the new preference but also preserve its original capabilities after customization. Drawing inspiration from the observation that human preference can be expressed as a reward model, we propose to cast LLM customization as optimizing the sum of two reward functions, one of which (denoted as $r_1$) was used to pre-train the LLM while the other (denoted as $r_2$) characterizes the new human preference. The obstacle here is that both reward functions are unknown, making the application of modern reinforcement learning methods infeasible. Thanks to the residual Q-learning framework, we can restore the customized LLM with the pre-trained LLM and the \emph{residual Q-function} without the reward function $r_1$. Moreover, we find that for a fixed pre-trained LLM, the reward function $r_2$ can be derived from the residual Q-function, enabling us to directly learn the residual Q-function from the new human preference data upon the Bradley-Terry model. We name our method Q-Adapter as it introduces an adapter module to approximate the residual Q-function for customizing the pre-trained LLM towards the new preference. Experiments based on the Llama-3.1 model on the DSP dataset and HH-RLHF dataset illustrate the superior effectiveness of Q-Adapter on both retaining existing knowledge and learning new preferences. Code is available at https://github.com/mansicer/Q-Adapter.

cs.LG↗

arXiv2025

LLMOPT: Learning to Define and Solve General Optimization Problems from Scratch

Optimization problems are prevalent across various scenarios. Formulating and then solving optimization problems described by natural language often requires highly specialized human expertise, which could block the widespread application of optimization-based decision making. To automate problem formulation and solving, leveraging large language models (LLMs) has emerged as a potential way. However, this kind of approach suffers from the issue of optimization generalization. Namely, the accuracy of most current LLM-based methods and the generality of optimization problem types that they can model are still limited. In this paper, we propose a unified learning-based framework called LLMOPT to boost optimization generalization. Starting from the natural language descriptions of optimization problems and a pre-trained LLM, LLMOPT constructs the introduced five-element formulation as a universal model for learning to define diverse optimization problem types. Then, LLMOPT employs the multi-instruction tuning to enhance both problem formalization and solver code generation accuracy and generality. After that, to prevent hallucinations in LLMs, such as sacrificing solving accuracy to avoid execution errors, the model alignment and self-correction mechanism are adopted in LLMOPT. We evaluate the optimization generalization ability of LLMOPT and compared methods across six real-world datasets covering roughly 20 fields such as health, environment, energy and manufacturing, etc. Extensive experiment results show that LLMOPT is able to model various optimization problem types such as linear/nonlinear programming, mixed integer programming, and combinatorial optimization, and achieves a notable 11.08% average solving accuracy improvement compared with the state-of-the-art methods. The code is available at https://github.com/caigaojiang/LLMOPT.

cs.AI↗

arXiv2025

Physics-Aware Inverse Design for Nanowire Single-Photon Avalanche Detectors via Deep Learning

Single-photon avalanche detectors (SPADs) have enabled various applications in emerging photonic quantum information technologies in recent years. However, despite many efforts to improve SPAD's performance, the design of SPADs remained largely an iterative and time-consuming process where a designer makes educated guesses of a device structure based on empirical reasoning and solves the semiconductor drift-diffusion model for it. In contrast, the inverse problem, i.e., directly inferring a structure needed to achieve desired performance, which is of ultimate interest to designers, remains an unsolved problem. We propose a novel physics-aware inverse design workflow for SPADs using a deep learning model and demonstrate it with an example of finding the key parameters of semiconductor nanowires constituting the unit cell of an SPAD, given target photon detection efficiency. Our inverse design workflow is not restricted to the case demonstrated and can be applied to design conventional planar structure-based SPADs, photodetectors, and solar cells.

physics.app-ph↗

arXiv2025

InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning

Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) have made significant advancements in reasoning capabilities. However, they still face challenges such as high computational demands and privacy concerns. This paper focuses on developing efficient Small Language Models (SLMs) and Multimodal Small Language Models (MSLMs) that retain competitive reasoning abilities. We introduce a novel training pipeline that enhances reasoning capabilities and facilitates deployment on edge devices, achieving state-of-the-art performance while minimizing development costs. \InfR~ aims to advance AI systems by improving reasoning, reducing adoption barriers, and addressing privacy concerns through smaller model sizes. Resources are available at https://github. com/Reallm-Labs/InfiR.

cs.CL↗

arXiv2025

Liquid Metal-Exfoliated SnO$_2$-Based Mixed-dimensional Heterostructures for Visible-to-Near-Infrared Photodetection

Ultra-thin two-dimensional (2D) materials have gained significant attention for making next-generation optoelectronic devices. Here, we report a large-area heterojunction photodetector fabricated using a liquid metal-printed 2D $\text{SnO}_2$ layer transferred onto CdTe thin films. The resulting device demonstrates efficient broadband light sensing from visible to near-infrared wavelengths, with enhanced detectivity and faster photo response than bare CdTe photodetectors. Significantly, the device shows a nearly $10^5$-fold increase in current than the dark current level when illuminated with a 780 nm laser and achieves a specific detectivity of around $10^{12} \, \text{Jones}$, nearly two orders of magnitude higher than a device with pure CdTe thin film. Additionally, temperature-dependent optoelectronic testing shows that the device maintains a stable response up to $140^\circ \text{C}$ and generates distinctive photocurrent at temperatures up to $80^\circ \text{C}$, demonstrating its thermal stability. Using band structure analysis, density functional theory (DFT) calculations, and photocurrent mapping, the formation of a $p$-$n$ junction is indicated, contributing to the enhanced photo response attributed to the efficient carrier separation by the built-in potential in the hetero-junction and the superior electron mobility of 2D $\text{SnO}_2$. Our results highlight the effectiveness of integrating liquid metal-exfoliated 2D materials for enhanced photodetector performance.

cond-mat.mtrl-sci↗

arXiv2025

Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning

As a marriage between offline RL and meta-RL, the advent of offline meta-reinforcement learning (OMRL) has shown great promise in enabling RL agents to multi-task and quickly adapt while acquiring knowledge safely. Among which, context-based OMRL (COMRL) as a popular paradigm, aims to learn a universal policy conditioned on effective task representations. In this work, by examining several key milestones in the field of COMRL, we propose to integrate these seemingly independent methodologies into a unified framework. Most importantly, we show that the pre-existing COMRL algorithms are essentially optimizing the same mutual information objective between the task variable $M$ and its latent representation $Z$ by implementing various approximate bounds. Such theoretical insight offers ample design freedom for novel algorithms. As demonstrations, we propose a supervised and a self-supervised implementation of $I(Z; M)$, and empirically show that the corresponding optimization algorithms exhibit remarkable generalization across a broad spectrum of RL benchmarks, context shift scenarios, data qualities and deep learning architectures. This work lays the information theoretic foundation for COMRL methods, leading to a better understanding of task representation learning in the context of reinforcement learning. Given its generality, we envision our framework as a promising offline pre-training paradigm of foundation models for decision making.

cs.LG↗

arXiv2025

Multi-Purpose Architecture for Fast Reset and Protective Readout of Superconducting Qubits

The ability to fast reset a qubit state is crucial for quantum information processing. However, to actively reset a qubit requires engineering a pathway to interact with a dissipative bath, which often comes with the cost of reduced qubit protection from the environment. Here, we present a novel multi-purpose architecture that enables fast reset and protection of superconducting qubits during control and readout. In our design, two on-chip diplexers are connected by two transmission lines. The high-pass branch provides a flat passband for convenient allocation of readout resonators above the qubit frequencies, which is preferred for reducing measurement-induced state transitions. In the low-pass branch, we leverage a standing-wave mode below the maximum qubit frequency for a rapid reset. The qubits are located in the common stopband to inhibit dissipation during coherent operations. We demonstrate resetting a transmon qubit from its first excited state to the ground state in 100 ns, achieving a residual population of 2.7%. The reset time may be further shortened to 27 ns by exploiting the coherent population inversion effect. We further extend the technique to resetting the qubit from its second excited state. Our approach promises scalable implementation of fast reset and qubit protection during control and readout, adding to the toolbox of dissipation engineering.

quant-ph↗

arXiv2025

MPT: A Large-scale Multi-Phytoplankton Tracking Benchmark

Phytoplankton are a crucial component of aquatic ecosystems, and effective monitoring of them can provide valuable insights into ocean environments and ecosystem changes. Traditional phytoplankton monitoring methods are often complex and lack timely analysis. Therefore, deep learning algorithms offer a promising approach for automated phytoplankton monitoring. However, the lack of large-scale, high-quality training samples has become a major bottleneck in advancing phytoplankton tracking. In this paper, we propose a challenging benchmark dataset, Multiple Phytoplankton Tracking (MPT), which covers diverse background information and variations in motion during observation. The dataset includes 27 species of phytoplankton and zooplankton, 14 different backgrounds to simulate diverse and complex underwater environments, and a total of 140 videos. To enable accurate real-time observation of phytoplankton, we introduce a multi-object tracking method, Deviation-Corrected Multi-Scale Feature Fusion Tracker(DSFT), which addresses issues such as focus shifts during tracking and the loss of small target information when computing frame-to-frame similarity. Specifically, we introduce an additional feature extractor to predict the residuals of the standard feature extractor's output, and compute multi-scale frame-to-frame similarity based on features from different layers of the extractor. Extensive experiments on the MPT have demonstrated the validity of the dataset and the superiority of DSFT in tracking phytoplankton, providing an effective solution for phytoplankton monitoring.

cs.CV↗

arXiv2024

Heterogeneous Multi-agent Zero-Shot Coordination by Coevolution

Generating agents that can achieve zero-shot coordination (ZSC) with unseen partners is a new challenge in cooperative multi-agent reinforcement learning (MARL). Recently, some studies have made progress in ZSC by exposing the agents to diverse partners during the training process. They usually involve self-play when training the partners, implicitly assuming that the tasks are homogeneous. However, many real-world tasks are heterogeneous, and hence previous methods may be inefficient. In this paper, we study the heterogeneous ZSC problem for the first time and propose a general method based on coevolution, which coevolves two populations of agents and partners through three sub-processes: pairing, updating and selection. Experimental results on various heterogeneous tasks highlight the necessity of considering the heterogeneous setting and demonstrate that our proposed method is a promising solution for heterogeneous ZSC tasks.

cs.NE↗

arXiv2024

A structure-preserving collisional particle method for the Landau kinetic equation

In this paper, we propose and implement a structure-preserving stochastic particle method for the Landau equation. The method is based on a particle system for the Landau equation, where pairwise grazing collisions are modeled as diffusion processes. By exploiting the unique structure of the particle system and a spherical Brownian motion sampling, the method avoids additional temporal discretization of the particle system, ensuring that the discrete-time particle distributions exactly match their continuous-time counterparts. The method achieves $O(N)$ complexity per time step and preserves fundamental physical properties, including the conservation of mass, momentum and energy, as well as entropy dissipation. It demonstrates strong long-time accuracy and stability in numerical experiments. Furthermore, we also apply the method to the spatially non-homogeneous equations through a case study of the Vlasov--Poisson--Landau equation.

math.NA↗

arXiv2024

Force chains bias the dynamic response to impacts in rubble-pile asteroids

The impact response of rubble-pile asteroids is essential for both elucidating their formation and evolution history and evaluating the efficacy of impact defense strategies. Although state-of-the-art numerical simulations have allowed for the replication of many macroscopic impact characteristics consistent with observations, the understanding of dynamics and response mechanisms within rubble-pile structures remains incomplete and requires further in-depth investigation. Such understanding is critical for assessing the effects and safety of impact defense missions. The loose structure of rubble-pile asteroids affects inhomogeneous internal stress propagation via inherent force chains, which may lead to structural fracturing. We demonstrate this phenomenon here, using a proof-of-principle two-dimensional model of granular aggregates. We find that the velocity response front to impact disturbances preferentially propagates along pre-existing force chains, with particles not in chains responding more slowly. The sites within the response zone where high dynamic stresses manifest are strongly correlated with these initial force chains, and the damages that result are predominantly located within areas enclosed by these chains. The strong correlation between pre-existing force chains and dynamic response is independent of the location, magnitude, direction of the disturbance velocity, or the aggregate's particle size distribution. All evidence suggests that the core reasons for this propagation preference lie in the structural heterogeneity of granular aggregates and the resulting differences in mechanical wave propagation. This investigation provides guidance for future research aimed at quantitatively assessing fragmentation risks based on the statistical properties of force chains.

astro-ph.EP↗

arXiv2024

Minimal pole representation and analytic continuation of matrix-valued correlation functions

We present a minimal pole method for analytically continuing matrix-valued imaginary frequency correlation functions to the real axis, enabling precise access to off-diagonal elements and thus improving the interpretation of self-energies and susceptibilities in quantum simulations. Traditional methods for matrix-valued analytic continuation tend to be either noise-sensitive or make ad-hoc positivity assumptions. Our approach avoides these issues via the construction of a compact pole representation with shared poles through exponential fits, expanding upon prior work focused on scalar functions. We test our method across various scenarios, including fermionic and bosonic response functions, with and without noise, and for both continuous and discrete spectra of real materials and model systems. Our findings demonstrate that this technique addresses the shortcomings of existing methodologies, such as artificial broadening and positivity violations. The paper is supplemented with a sample implementation in Python.

cond-mat.str-el↗

arXiv2024

POINTS1.5: Building a Vision-Language Model towards Real World Applications

Vision-language models have made significant strides recently, demonstrating superior performance across a range of tasks, e.g. optical character recognition and complex diagram analysis. Building on this trend, we introduce a new vision-language model, POINTS1.5, designed to excel in various real-world applications. POINTS1.5 is an enhancement of POINTS1.0 and incorporates several key innovations: i) We replace the original CLIP vision encoder, which had a fixed image resolution, with a NaViT-style vision encoder that supports native dynamic high resolution. This allows POINTS1.5 to process images of any resolution without needing to split them into tiles. ii) We add bilingual support to POINTS1.5, significantly enhancing its capability in Chinese. Due to the scarcity of open-source Chinese datasets for vision-language models, we collect numerous images from the Internet and annotate them using a combination of manual and automatic methods. iii) We propose a set of rigorous filtering methods for visual instruction tuning datasets. We comprehensively evaluate all these filtering methods, and choose the most effective ones to obtain the final visual instruction tuning set. Thanks to these innovations, POINTS1.5 significantly outperforms POINTS1.0 and demonstrates strong performance across a range of real-world applications. Notably, POINTS1.5-7B is trained on fewer than 4 billion tokens and ranks first on the OpenCompass leaderboard among models with fewer than 10 billion parameters

cs.CV↗

arXiv2024

Universal and Context-Independent Triggers for Precise Control of LLM Outputs

Large language models (LLMs) have been widely adopted in applications such as automated content generation and even critical decision-making systems. However, the risk of prompt injection allows for potential manipulation of LLM outputs. While numerous attack methods have been documented, achieving full control over these outputs remains challenging, often requiring experienced attackers to make multiple attempts and depending heavily on the prompt context. Recent advancements in gradient-based white-box attack techniques have shown promise in tasks like jailbreaks and system prompt leaks. Our research generalizes gradient-based attacks to find a trigger that is (1) Universal: effective irrespective of the target output; (2) Context-Independent: robust across diverse prompt contexts; and (3) Precise Output: capable of manipulating LLM inputs to yield any specified output with high accuracy. We propose a novel method to efficiently discover such triggers and assess the effectiveness of the proposed attack. Furthermore, we discuss the substantial threats posed by such attacks to LLM-based applications, highlighting the potential for adversaries to taking over the decisions and actions made by AI agents.

cs.CL↗

arXiv2024

Stable Continual Reinforcement Learning via Diffusion-based Trajectory Replay

Given the inherent non-stationarity prevalent in real-world applications, continual Reinforcement Learning (RL) aims to equip the agent with the capability to address a series of sequentially presented decision-making tasks. Within this problem setting, a pivotal challenge revolves around \textit{catastrophic forgetting} issue, wherein the agent is prone to effortlessly erode the decisional knowledge associated with past encountered tasks when learning the new one. In recent progresses, the \textit{generative replay} methods have showcased substantial potential by employing generative models to replay data distribution of past tasks. Compared to storing the data from past tasks directly, this category of methods circumvents the growing storage overhead and possible data privacy concerns. However, constrained by the expressive capacity of generative models, existing \textit{generative replay} methods face challenges in faithfully reconstructing the data distribution of past tasks, particularly in scenarios with a myriad of tasks or high-dimensional data. Inspired by the success of diffusion models in various generative tasks, this paper introduces a novel continual RL algorithm DISTR (Diffusion-based Trajectory Replay) that employs a diffusion model to memorize the high-return trajectory distribution of each encountered task and wakeups these distributions during the policy learning on new tasks. Besides, considering the impracticality of replaying all past data each time, a prioritization mechanism is proposed to prioritize the trajectory replay of pivotal tasks in our method. Empirical experiments on the popular continual RL benchmark \texttt{Continual World} demonstrate that our proposed method obtains a favorable balance between \textit{stability} and \textit{plasticity}, surpassing various existing continual RL baselines in average success rate.

cs.LG↗

arXiv2024

PhyTracker: An Online Tracker for Phytoplankton

Phytoplankton, a crucial component of aquatic ecosystems, requires efficient monitoring to understand marine ecological processes and environmental conditions. Traditional phytoplankton monitoring methods, relying on non-in situ observations, are time-consuming and resource-intensive, limiting timely analysis. To address these limitations, we introduce PhyTracker, an intelligent in situ tracking framework designed for automatic tracking of phytoplankton. PhyTracker overcomes significant challenges unique to phytoplankton monitoring, such as constrained mobility within water flow, inconspicuous appearance, and the presence of impurities. Our method incorporates three innovative modules: a Texture-enhanced Feature Extraction (TFE) module, an Attention-enhanced Temporal Association (ATA) module, and a Flow-agnostic Movement Refinement (FMR) module. These modules enhance feature capture, differentiate between phytoplankton and impurities, and refine movement characteristics, respectively. Extensive experiments on the PMOT dataset validate the superiority of PhyTracker in phytoplankton tracking, and additional tests on the MOT dataset demonstrate its general applicability, outperforming conventional tracking methods. This work highlights key differences between phytoplankton and traditional objects, offering an effective solution for phytoplankton monitoring.

cs.CV↗