arXiv subjects

Yi Wu

Publications and source records attributed to Yi Wu.

At least 109 records · Page 6Linked to original sources

arXiv2025

Large ferromagnetic-like band splitting in ultrathin ${\mathrm{SmC}}_{6}$ films

Two-dimensional (2D) magnetic materials provide a unique platform for exploring quantum phases from magnetic order in reduced dimensions. While there have been extensive studies on 2D magnetic materials based on 3$d$ electrons, experimental studies on 4$f$-electron counterparts are far fewer, particularly on their electronic structure. In this study, we report the successful synthesis of ultrathin ${\mathrm{SmC}}_{6}$ films using molecular beam epitaxy. Utilizing in situ angle-resolved photoemission spectroscopy (ARPES), we uncover a large band splitting in the valence bands, which we attribute to the ferromagnetic order driven by exchange couplings between Sm 4$f$ moments and conduction electrons. Despite the small magnetic moment of Sm, the observed splitting is comparable to those of Eu- and Gd-based systems with much larger local moments. Interestingly, the surface state also exhibits splitting with similar magnitude and can be eliminated by overannealing, while the valence bands with ferromagnetic-like splittings remain robust. Our work provides spectroscopic insight to understand the electronic origin of magnetic order in Sm-based compounds. Our study also offers a platform to study 2D magnetic materials based on 4$f$ electrons.

cond-mat.str-el↗

arXiv2025

Reflective VLM Planning for Dual-Arm Desktop Cleaning: Bridging Open-Vocabulary Perception and Precise Manipulation

Desktop cleaning demands open-vocabulary recognition and precise manipulation for heterogeneous debris. We propose a hierarchical framework integrating reflective Vision-Language Model (VLM) planning with dual-arm execution via structured scene representation. Grounded-SAM2 facilitates open-vocabulary detection, while a memory-augmented VLM generates, critiques, and revises manipulation sequences. These sequences are converted into parametric trajectories for five primitives executed by coordinated Franka arms. Evaluated in simulated scenarios, our system achieving 87.2% task completion, a 28.8% improvement over static VLM and 36.2% over single-arm baselines. Structured memory integration proves crucial for robust, generalizable manipulation while maintaining real-time control performance.

cs.RO↗

arXiv2025

Learning Strategic Language Agents in the Werewolf Game with Iterative Latent Space Policy Optimization

Large language model (LLM) agents have recently demonstrated impressive capabilities in various domains like open-ended conversation and multi-step decision-making. However, it remains challenging for these agents to solve strategic language games, such as Werewolf, which demand both strategic decision-making and free-form language interactions. Existing LLM agents often suffer from intrinsic bias in their action distributions and limited exploration of the unbounded text action space, resulting in suboptimal performance. To address these challenges, we propose Latent Space Policy Optimization (LSPO), an iterative framework that combines game-theoretic methods with LLM fine-tuning to build strategic language agents. LSPO leverages the observation that while the language space is combinatorially large, the underlying strategy space is relatively compact. We first map free-form utterances into a finite latent strategy space, yielding an abstracted extensive-form game. Then we apply game-theoretic methods like Counterfactual Regret Minimization (CFR) to optimize the policy in the latent space. Finally, we fine-tune the LLM via Direct Preference Optimization (DPO) to align with the learned policy. By iteratively alternating between these steps, our LSPO agents progressively enhance both strategic reasoning and language communication. Experiment on the Werewolf game shows that our agents iteratively expand the strategy space with improving performance and outperform existing Werewolf agents, underscoring their effectiveness in free-form language games with strategic interactions.

cs.AI↗

arXiv2025

Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game

Agents built with large language models (LLMs) have shown great potential across a wide range of domains. However, in complex decision-making tasks, pure LLM-based agents tend to exhibit intrinsic bias in their choice of actions, which is inherited from the model's training data and results in suboptimal performance. To develop strategic language agents, i.e., agents that generate flexible language actions and possess strong decision-making abilities, we propose a novel framework that powers LLM-based agents with reinforcement learning (RL). We consider Werewolf, a popular social deduction game, as a challenging testbed that emphasizes versatile communication and strategic gameplay. To mitigate the intrinsic bias in language actions, our agents use an LLM to perform deductive reasoning and generate a diverse set of action candidates. Then an RL policy trained to optimize the decision-making ability chooses an action from the candidates to play in the game. Extensive experiments show that our agents overcome the intrinsic bias and outperform existing LLM-based agents in the Werewolf game. We also conduct human-agent experiments and find that our agents achieve human-level performance and demonstrate strong strategic play.

cs.AI↗

arXiv2025

StyleAR: Customizing Multimodal Autoregressive Model for Style-Aligned Text-to-Image Generation

In the current research landscape, multimodal autoregressive (AR) models have shown exceptional capabilities across various domains, including visual understanding and generation. However, complex tasks such as style-aligned text-to-image generation present significant challenges, particularly in data acquisition. In analogy to instruction-following tuning for image editing of AR models, style-aligned generation requires a reference style image and prompt, resulting in a text-image-to-image triplet where the output shares the style and semantics of the input. However, acquiring large volumes of such triplet data with specific styles is considerably more challenging than obtaining conventional text-to-image data used for training generative models. To address this issue, we propose StyleAR, an innovative approach that combines a specially designed data curation method with our proposed AR models to effectively utilize text-to-image binary data for style-aligned text-to-image generation. Our method synthesizes target stylized data using a reference style image and prompt, but only incorporates the target stylized image as the image modality to create high-quality binary data. To facilitate binary data training, we introduce a CLIP image encoder with a perceiver resampler that translates the image input into style tokens aligned with multimodal tokens in AR models and implement a style-enhanced token technique to prevent content leakage which is a common issue in previous work. Furthermore, we mix raw images drawn from large-scale text-image datasets with stylized images to enhance StyleAR's ability to extract richer stylistic features and ensure style consistency. Extensive qualitative and quantitative experiments demonstrate our superior performance.

cs.CV↗

arXiv2025

Neural Internal Model Control: Learning a Robust Control Policy via Predictive Error Feedback

Accurate motion control in the face of disturbances within complex environments remains a major challenge in robotics. Classical model-based approaches often struggle with nonlinearities and unstructured disturbances, while RL-based methods can be fragile when encountering unseen scenarios. In this paper, we propose a novel framework, Neural Internal Model Control, which integrates model-based control with RL-based control to enhance robustness. Our framework streamlines the predictive model by applying Newton-Euler equations for rigid-body dynamics, eliminating the need to capture complex high-dimensional nonlinearities. This internal model combines model-free RL algorithms with predictive error feedback. Such a design enables a closed-loop control structure to enhance the robustness and generalizability of the control system. We demonstrate the effectiveness of our framework on both quadrotors and quadrupedal robots, achieving superior performance compared to state-of-the-art methods. Furthermore, real-world deployment on a quadrotor with rope-suspended payloads highlights the framework's robustness in sim-to-real transfer. Our code is released at https://github.com/thu-uav/NeuralIMC.

cs.RO↗

arXiv2025

What Matters in Learning A Zero-Shot Sim-to-Real RL Policy for Quadrotor Control? A Comprehensive Study

Executing precise and agile flight maneuvers is critical for quadrotors in various applications. Traditional quadrotor control approaches are limited by their reliance on flat trajectories or time-consuming optimization, which restricts their flexibility. Recently, RL-based policy has emerged as a promising alternative due to its ability to directly map observations to actions, reducing the need for detailed system knowledge and actuation constraints. However, a significant challenge remains in bridging the sim-to-real gap, where RL-based policies often experience instability when deployed in real world. In this paper, we investigate key factors for learning robust RL-based control policies that are capable of zero-shot deployment in real-world quadrotors. We identify five critical factors and we develop a PPO-based training framework named SimpleFlight, which integrates these five techniques. We validate the efficacy of SimpleFlight on Crazyflie quadrotor, demonstrating that it achieves more than a 50% reduction in trajectory tracking error compared to state-of-the-art RL baselines. The policy derived by SimpleFlight consistently excels across both smooth polynominal trajectories and challenging infeasible zigzag trajectories on small thrust-to-weight quadrotors. In contrast, baseline methods struggle with high-speed or infeasible trajectories. To support further research and reproducibility, we integrate SimpleFlight into a GPU-based simulator Omnidrones and provide open-source access to the code and model checkpoints. We hope SimpleFlight will offer valuable insights for advancing RL-based quadrotor control. For more details, visit our project website at https://sites.google.com/view/simpleflight/.

cs.RO↗

arXiv2025

$3d$ flat bands and coupled $4f$ moments in the kagome-honeycomb permanent magnet Sm$_{2}$Co$_{17}$

Rare earth permanent magnets (REPMs) with both localized moments and itinerant conduction bands are not only important for fundamental research but also have significant technological applications. In particular, Sm$_{\rm 2}$Co$_{\rm 17}$ is a prototypical high-temperture REPM, where the Co atoms form a kagome-honeycomb stacked lattice. Here we report synthesis of epitaxial Sm$_{\rm 2}$Co$_{\rm 17}$ films using molecular beam epitaxy and measurements of their momentum-resolved electronic structure from \textit{in-situ} angle-resolved photoemission spectroscopy. Our results unveil two flat bands from Co $3d$ orbitals near the Fermi level ($E_F$), one at $\sim$\,--300\,meV and another right at $E_F$, which arise from orbital-selective destructive interference and strong electron correlations, respectively. In addition, our results reveal that Sm $4f$ states are far away from $E_F$ (hence mostly localized) and exhibit an anomalous temperature dependence, caused by the $3d$-$4f$ magnetic coupling. Our findings provide direct spectroscopic insights to understand the strong uniaxial ferromagnetism in Sm$_{\rm 2}$Co$_{\rm 17}$ (and REPMs alike). Our work also opens avenues to explore flat-band physics near $E_F$ and emergent phenomena in correlated kagome-honeycomb lattices.

cond-mat.str-el↗

arXiv2025

FlightBench: Benchmarking Learning-based Methods for Ego-vision-based Quadrotors Navigation

Ego-vision-based navigation in cluttered environments is crucial for mobile systems, particularly agile quadrotors. While learning-based methods have shown promise recently, head-to-head comparisons with cutting-edge optimization-based approaches are scarce, leaving open the question of where and to what extent they truly excel. In this paper, we introduce FlightBench, the first comprehensive benchmark that implements various learning-based methods for ego-vision-based navigation and evaluates them against mainstream optimization-based baselines using a broad set of performance metrics. More importantly, we develop a suite of criteria to assess scenario difficulty and design test cases that span different levels of difficulty based on these criteria. Our results show that while learning-based methods excel in high-speed flight and faster inference, they struggle with challenging scenarios like sharp corners or view occlusion. Analytical experiments validate the correlation between our difficulty criteria and flight performance. Moreover, we verify the trend in flight performance within real-world environments through full-pipeline and hardware-in-the-loop experiments. We hope this benchmark and these criteria will drive future advancements in learning-based navigation for ego-vision quadrotors. Code and documentation are available at https://github.com/thu-uav/FlightBench.

cs.RO↗

arXiv2025

PaRT: Enhancing Proactive Social Chatbots with Personalized Real-Time Retrieval

Social chatbots have become essential intelligent companions in daily scenarios ranging from emotional support to personal interaction. However, conventional chatbots with passive response mechanisms usually rely on users to initiate or sustain dialogues by bringing up new topics, resulting in diminished engagement and shortened dialogue duration. In this paper, we present PaRT, a novel framework enabling context-aware proactive dialogues for social chatbots through personalized real-time retrieval and generation. Specifically, PaRT first integrates user profiles and dialogue context into a large language model (LLM), which is initially prompted to refine user queries and recognize their underlying intents for the upcoming conversation. Guided by refined intents, the LLM generates personalized dialogue topics, which then serve as targeted queries to retrieve relevant passages from RedNote. Finally, we prompt LLMs with summarized passages to generate knowledge-grounded and engagement-optimized responses. Our approach has been running stably in a real-world production environment for more than 30 days, achieving a 21.77\% improvement in the average duration of dialogues.

cs.AI↗

arXiv2025

Engineering Graphene Nanoribbons via Periodically Embedding Oxygen Atoms

Heteroatom doping is an important method for engineering graphene nanoribbons (GNRs) because of its ability to modify electronic properties by introducing extra electrons or vacancies. However, precisely integrating oxygen atoms into the lattice of GNRs is unexplored, and the resulting electronic properties remain elusive. Here, we achieve the precise embedding of oxygen atoms into the lattice of GNRs via in situ formation of pyrans, synthesizing two types of oxygen-doped GNRs (O-doped chevron-GNR and O-doped chiral (2,1)-GNR). Using scanning tunneling microscopy, non-contact atomic force microscopy, and density functional theory calculations, the atomic structures and electronic properties of O-doped GNRs are determined, demonstrating that both GNRs are direct bandgap semiconductors with different sensitivities to oxygen dopants. Oxygen dopants have a minor impact on the bandgap of chevron-GNR but a significant effect on the bandgap of chiral (2,1)-GNR, which is attributed to the difference in density of states near the Fermi level between substituted intrinsic carbon atoms and their pristine counterparts. Compared with the pristine chiral (2,1)-GNR, the band structure of O-doped chiral (2,1)-GNR exhibits unexpected band edges transition, which is ascribed to sp2-hybridized oxygen atoms which introduces additional electrons to the conduction band of chiral (2,1)-GNR, leading to the upward shift of Fermi surface.

cond-mat.mtrl-sci↗

arXiv2025

ReaL: Efficient RLHF Training of Large Language Models with Parameter Reallocation

Reinforcement Learning from Human Feedback (RLHF) is a pivotal technique for empowering large language model (LLM) applications. Compared with the supervised training process of LLMs, the RLHF training process is much more sophisticated, requiring a diverse range of computation workloads with intricate dependencies between multiple LLM instances. Therefore, simply adopting the fixed parallelization strategies from supervised training for LLMs can be insufficient for RLHF and result in low training efficiency. To overcome this limitation, we propose a novel technique named parameter ReaLlocation, which dynamically adapts the parallelization strategies for different workloads during training by redistributing LLM parameters across the training cluster. Building upon this idea, we introduce ReaL, a pioneering system for efficient RLHF training. ReaL introduces the concept of an execution plan, which defines a fine-grained resource allocation and parallelization strategy particularly designed for RLHF training. Based on this concept, ReaL employs a tailored search algorithm with a lightweight run-time estimator to automatically discover an efficient execution plan for an instance of RLHF experiment. Subsequently, the runtime engine deploys the selected plan by effectively parallelizing computations and redistributing parameters. We evaluate ReaL on the LLaMA models with up to 70 billion parameters and 128 GPUs. The experimental results demonstrate that ReaL achieves speedups of up to $3.58\times$ compared to baseline methods. Furthermore, the execution plans generated by ReaL exhibit an average of $81\%$ performance improvement over heuristic approaches based on Megatron-LM in the long-context scenario. The source code of ReaL is publicly available at https://github.com/openpsi-project/ReaLHF .

cs.DC↗

arXiv2025

Low energy electrodynamics and a hidden Fermi liquid in the heavy-fermion CeCoIn$_5$

We present time-domain THz spectroscopy of thin films of the heavy-fermion superconductor CeCoIn$_5$. Below the $\approx$ 40 K Kondo coherence temperature, a narrow Drude-like peak forms, as the result of the $f$ orbital - conduction electron hybridization and the formation of the heavy-fermion state. The complex optical conductivity is analyzed through a Drude model and extended Drude model analysis. Via the extended Drude model analysis, we measure the frequency-dependent scattering rate ($1/ τ$) and effective mass ($m^*/m_b$). This scattering rate shows a linear dependence on temperature, which matches the dependence of the resistivity as expected. Nevertheless, the width of the low-frequency Drude peak itself that is set by the {\it renormalized} quasiparticle scattering rate ($1 / τ^* = m_b/ m^* τ$) shows a $T^2$ dependence. This is the scattering rate that characterizes the relaxation time of the renormalized quasiparticles. This gives evidence for Fermi liquid state, which in conventional transport experiments is hidden by the strong temperature dependent mass.

cond-mat.str-el↗

arXiv2025

ICPL: Few-shot In-context Preference Learning via LLMs

Preference-based reinforcement learning is an effective way to handle tasks where rewards are hard to specify but can be exceedingly inefficient as preference learning is often tabula rasa. We demonstrate that Large Language Models (LLMs) have native preference-learning capabilities that allow them to achieve sample-efficient preference learning, addressing this challenge. We propose In-Context Preference Learning (ICPL), which uses in-context learning capabilities of LLMs to reduce human query inefficiency. ICPL uses the task description and basic environment code to create sets of reward functions which are iteratively refined by placing human feedback over videos of the resultant policies into the context of an LLM and then requesting better rewards. We first demonstrate ICPL's effectiveness through a synthetic preference study, providing quantitative evidence that it significantly outperforms baseline preference-based methods with much higher performance and orders of magnitude greater efficiency. We observe that these improvements are not solely coming from LLM grounding in the task but that the quality of the rewards improves over time, indicating preference learning capabilities. Additionally, we perform a series of real human preference-learning trials and observe that ICPL extends beyond synthetic settings and can work effectively with humans-in-the-loop.

cs.AI↗

arXiv2025

Long-horizon Locomotion and Manipulation on a Quadrupedal Robot with Large Language Models

We present a large language model (LLM) based system to empower quadrupedal robots with problem-solving abilities for long-horizon tasks beyond short-term motions. Long-horizon tasks for quadrupeds are challenging since they require both a high-level understanding of the semantics of the problem for task planning and a broad range of locomotion and manipulation skills to interact with the environment. Our system builds a high-level reasoning layer with large language models, which generates hybrid discrete-continuous plans as robot code from task descriptions. It comprises multiple LLM agents: a semantic planner that sketches a plan, a parameter calculator that predicts arguments in the plan, a code generator that converts the plan into executable robot code, and a replanner that handles execution failures or human interventions. At the low level, we adopt reinforcement learning to train a set of motion planning and control skills to unleash the flexibility of quadrupeds for rich environment interactions. Our system is tested on long-horizon tasks that are infeasible to complete with one single skill. Simulation and real-world experiments show that it successfully figures out multi-step strategies and demonstrates non-trivial behaviors, including building tools or notifying a human for help. Demos are available on our project page: https://sites.google.com/view/long-horizon-robot.

cs.RO↗

arXiv2025

Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models

State Space Models (SSMs) have emerged as a promising alternative to the popular transformer-based models and have been increasingly gaining attention. Compared to transformers, SSMs excel at tasks with sequential data or longer contexts, demonstrating comparable performances with significant efficiency gains. In this survey, we provide a coherent and systematic overview for SSMs, including their theoretical motivations, mathematical formulations, comparison with existing model classes, and various applications. We divide the SSM series into three main sections, providing a detailed introduction to the original SSM, the structured SSM represented by S4, and the selective SSM typified by Mamba. We put an emphasis on technicality, and highlight the various key techniques introduced to address the effectiveness and efficiency of SSMs. We hope this manuscript serves as an introduction for researchers to explore the theoretical foundations of SSMs.

cs.LG↗

arXiv2025

Leveraging Symmetry in RL-based Legged Locomotion Control

Model-free reinforcement learning is a promising approach for autonomously solving challenging robotics control problems, but faces exploration difficulty without information of the robot's kinematics and dynamics morphology. The under-exploration of multiple modalities with symmetric states leads to behaviors that are often unnatural and sub-optimal. This issue becomes particularly pronounced in the context of robotic systems with morphological symmetries, such as legged robots for which the resulting asymmetric and aperiodic behaviors compromise performance, robustness, and transferability to real hardware. To mitigate this challenge, we can leverage symmetry to guide and improve the exploration in policy learning via equivariance/invariance constraints. In this paper, we investigate the efficacy of two approaches to incorporate symmetry: modifying the network architectures to be strictly equivariant/invariant, and leveraging data augmentation to approximate equivariant/invariant actor-critics. We implement the methods on challenging loco-manipulation and bipedal locomotion tasks and compare with an unconstrained baseline. We find that the strictly equivariant policy consistently outperforms other methods in sample efficiency and task performance in simulation. In addition, symmetry-incorporated approaches exhibit better gait quality, higher robustness and can be deployed zero-shot in real-world experiments.

cs.RO↗

arXiv2025

Orientation-dependent superconductivity and electronic structure of the rare-earth metal/KTaO3 interfaces

The recent discovery of orientation-dependent superconductivity in KTaO3-based interfaces has attracted considerable interest, while the underlying origin remains an open question. Here we report a different approach to tune the interfacial electron gas and superconductivity by forming interfaces between rare-earth (RE) metals (RE being La, Ce, Eu) and KTaO3 substrates with different orientations. We found that the interfacial superconductivity is strongest for the Eu/KTaO3 interfaces, becomes weaker in La/KTaO3 and is absent in Ce/KTaO3. Using in-situ photoemission, we observed distinct valence bands associated with RE metals, as well as a pronounced orientation dependence in the interfacial electronic structure, which can be linked to the orientation-dependent superconductivity. The photoemission spectra show similar double-peak structures for the (111) and (110) oriented interfaces, with an energy separation close to the LO4 phonon of KTaO3. Detailed analyses suggest that this double-peak structure could be attributed to electron-phonon coupling, which might be important for the interfacial superconductivity.

cond-mat.supr-con↗