arXiv subjects

Yizhou Yang

Publications and source records attributed to Yizhou Yang.

13 recordsLinked to original sources

arXiv2026

StateLoop: Persistent State for Looped Transformers

Looped Transformers offer a parameter-efficient approach to language modeling by repeatedly applying a shared backbone. Vanilla looped Transformers use each backbone output directly as the input to the next iteration. These outputs may contain information useful several iterations later, but retaining it across the loop depends entirely on the single hidden state. We introduce StateLoop, which maintains one additional persistent state per token to retain information across iterations. At each iteration, a learned gate controls how much of the previous state is retained and how much information from the backbone output is incorporated. The updated state is projected back and added to the backbone output for subsequent iterations and final prediction. This design reduces Pile perplexity and improves average accuracy across eight zero-shot tasks over Vanilla Looped. Its perplexity advantage over Vanilla Looped grows with trained loop depth, consistent with persistent state becoming increasingly useful as recurrent computation deepens.

cs.CL↗

arXiv2026

LoopFold: Parallelizing Recurrent Depth in Looped Transformers

Looped Transformers increase computation per token by repeatedly applying a shared backbone, offering a way to improve language modeling and downstream task performance at a fixed backbone parameter count. However, each loop depends on the preceding loop, so increasing recurrent depth also increases decoding latency. Existing approaches include Parallel Loop Transformers, which reduce latency by overlapping loop computation across token positions, but yield rapidly diminishing marginal accuracy gains at larger depths. To address this challenge, we introduce LoopFold, which folds computation at multiple recurrent depths into parallel batches for the same token prefix. A lightweight predictor estimates inputs to later loops so they can start before earlier loops finish. A corrector then refines the outputs of these later loops using newly available results from earlier loops. All components are trained jointly using only the final next-token prediction loss. In experiments with 1.4B models, deeper LoopFold configurations achieve progressively higher average zero-shot accuracy across eight tasks, rising from 53.41% to 56.38%. Vanilla looped models and PLT show limited gains over the same depth range. Compared with a nine-loop vanilla model, LoopFold uses five sequential stages and additional parallel computation to reduce decoding latency by 46%.

cs.CL↗

arXiv2026

Diagnosing and Mitigating Evidence Misgrounding in Multimodal Knowledge Editing

Multimodal knowledge editing aims to update visually grounded facts in Multimodal Large Language Models (MLLMs) while preserving unrelated text and visual behaviors. However, standard answer-level metrics can hide evidence misgrounding: an edit may produce the target answer on the original image yet remain sensitive to incidental context rather than the intended visual evidence. To examine this issue in a controlled setting, we construct WaterBird-Edit, an analysis set that decouples bird habitat labels from background context and measures whether edited predictions remain stable when the target evidence is preserved but the background changes. Motivated by this diagnosis, we propose VICE (Visual In-Context Editing), a retrieval-based framework that stores edits in external memory and applies them through visual in-context demonstrations designed to expose task-relevant evidence. Experiments on three standard benchmarks show that VICE achieves strong lifelong editing performance, while results on WaterBird-Edit show that it substantially reduces sensitivity to the controlled background shifts. Code and dataset are available in https://github.com/Wang-Z-W/VICE-main.

cs.CV↗

arXiv2026

FisherEdit: Fisher Information Guided Knowledge Editing for Large Language Models

Sequential knowledge editing in Large Language Models (LLMs) often leads to a degradation of general capabilities due to cumulative parameter interference. In this paper, we revisit prominent projection-based methods like AlphaEdit and reveal a surprising mismatch: their robustness is primarily driven by update regularization rather than the intended null-space projection. Based on this insight, we propose FisherEdit, a theoretically grounded method built on Fisher geometry, which replaces coarse activation-space projections with importance-aware regularization in parameter space. It penalizes changes to sensitive parameters more strongly while retaining flexibility for new facts, and admits an efficient row-wise closed-form solution. Experiments on Llama-3, Qwen2.5, and Olmo-3 across multiple benchmarks demonstrate that FisherEdit achieves state-of-the-art performance while better preserving model stability and downstream task accuracy.

cs.CL↗

arXiv2026

BIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge Annotations

While recent Large Language Model (LLM)-based text-to-SQL systems achieve impressive performance on standard benchmarks, they struggle when user queries implicitly rely on domain-specific knowledge, such as business logic, data conventions, and analytical practices, that is neither captured by the schema nor explicitly stated in the natural language question. Historical SQL query logs offer a valuable source of such knowledge, yet existing benchmarks do not adequately support evaluation of history-driven approaches. To address this gap, we introduce BIRD-History, a benchmark consisting of 1,393 tasks across 11 databases, designed to evaluate text-to-SQL systems' ability to ground underspecified natural language questions using historical SQL scripts. Each task is annotated with ground-truth labels specifying which historical queries contain relevant knowledge and which SQL clauses encode it, enabling systematic evaluation of both retrieval effectiveness and knowledge utilization. Alongside the benchmark, we propose a plug-in retriever that extracts five types of external knowledge from historical SQL scripts, then retrieves and reranks relevant fragments for query generation. The retriever integrates seamlessly into existing few-shot text-to-SQL pipelines without requiring prompt modifications. Experiments demonstrate consistent improvements across four text-to-SQL systems, highlighting the value of leveraging historical query logs for handling underspecified queries. Dataset and code are open-sourced on https://github.com/zjuidg/BIRD-History.

cs.AI↗

arXiv2025

Engineering Social Optimality via Utility Shaping in Non-Cooperative Games under Incomplete Information and Imperfect Monitoring

In this paper, we study decentralized decision-making where agents optimize private objectives under incomplete information and imperfect public monitoring, in a non-cooperative setting. By shaping utilities-embedding shadow prices or Karush-Kuhn-Tucker(KKT)-aligned penalties-we make the stage game an exact-potential game whose unique equilibrium equals the (possibly constrained) social optimum. We characterize the Bayesian equilibrium as a stochastic variational inequality; strong monotonicity follows from a single-inflection compressed/stretched-exponential response combined with convex pricing. We give tracking bounds for damped-gradient and best-response-with-hysteresis updates under a noisy public index, and corresponding steady-state error. The framework accommodates discrete and continuous action sets and composes with slower discrete assignment. Deployable rules include: embed prices/penalties; publish a single public index; tune steps, damping, and dual rates for contraction. Computational experiments cover (i) a multi-tier supply chain and (ii) a non-cooperative agentic-AI compute market of bidding bots. Relative to price-only baselines, utility shaping attains near-centralized welfare, eliminates steady-state constraint/capacity violations when feasible, and accelerates convergence; with quantization, discrete equilibria track continuous ones within the mesh. The blueprint is portable to demand response, cloud/edge scheduling, and transportation pricing and biosecurity/agriculture. Overall, utility shaping plus a public index implements the constrained social optimum with stable equilibria under noise and drift-an operations-research-friendly alternative to heavy messaging or full mechanism design.

cs.GT↗

arXiv2025

LG-CD: Enhancing Language-Guided Change Detection through SAM2 Adaptation

Remote Sensing Change Detection (RSCD) typically identifies changes in land cover or surface conditions by analyzing multi-temporal images. Currently, most deep learning-based methods primarily focus on learning unimodal visual information, while neglecting the rich semantic information provided by multimodal data such as text. To address this limitation, we propose a novel Language-Guided Change Detection model (LG-CD). This model leverages natural language prompts to direct the network's attention to regions of interest, significantly improving the accuracy and robustness of change detection. Specifically, LG-CD utilizes a visual foundational model (SAM2) as a feature extractor to capture multi-scale pyramid features from high-resolution to low-resolution across bi-temporal remote sensing images. Subsequently, multi-layer adapters are employed to fine-tune the model for downstream tasks, ensuring its effectiveness in remote sensing change detection. Additionally, we design a Text Fusion Attention Module (TFAM) to align visual and textual information, enabling the model to focus on target change regions using text prompts. Finally, a Vision-Semantic Fusion Decoder (V-SFD) is implemented, which deeply integrates visual and semantic information through a cross-attention mechanism to produce highly accurate change detection masks. Our experiments on three datasets (LEVIR-CD, WHU-CD, and SYSU-CD) demonstrate that LG-CD consistently outperforms state-of-the-art change detection methods. Furthermore, our approach provides new insights into achieving generalized change detection by leveraging multimodal information.

cs.CV↗

arXiv2024

Room-temperature intrinsic ferromagnetism of two-dimensional Na2Cl crystals originated by s- and p-orbitals

Ferromagnetism, as one of the most valuable properties of materials, has attracted sustained and widespread interest in basic and applied research from ancient compasses to modern electronic devices. Traditionally, intrinsic ferromagnetism has been attributed to the permanent magnetic moment induced by partially filled d- or f-orbitals. However, the development of ferromagnetic materials has been limited by this electronic structure convention. Thus, the identification of additional materials that are not constrained by this conventional rule but also exhibit intrinsic ferromagnetism is highly expected and may impact all the fields based on ferromagnetism. Here, we report the direct observation of room-temperature intrinsic ferromagnetism in two-dimensional (2D) Na2Cl crystals, in which there are only partially filled s- and p-orbitals rather than d- or f-orbitals, using the superconducting quantum interference device (SQUID) and magnetic force microscope (MFM). These Na2Cl crystals formed in reduced graphene oxide (rGO) membranes have an unconventional stoichiometric structure leading to unique electron and spin distributions. And the structure of these 2D Na2Cl crystals, including the Na and Cl sites, is characterized in situ for the first time and directly observed by cryo-electron microscopy (cryo-EM) based on the observed difference in contrast between Na stacked with Cl and single Na. These findings break the conventional rule of intrinsic ferromagnetism and provide new insights into the design of novel magnetic and electronic devices and transistors with a size down to the atomic scale.

cond-mat.mtrl-sci↗

arXiv2022

Remarkably strong magnetic response in molecules with polar groups

For more than a century, electricity and magnetism have been believed to always exhibit inextricable link due to the symmetry in electromagnetism. At the interface, polar groups that have polar charges, are indispensable to be considered, which interact directly with other polar charges/external charges/external electric fields. However, there is no report on the corresponding magnetic properties on these polar groups. Clearly, such asymmetry, that is, only the interaction between the polar groups and charges, is out of bounds. Here we show that those molecules with considerable polar groups, such as cellulose acetate (CA) and other cellulose derivatives with different polar groups, can have strong magnetic response, indicating that they are strongly paramagnetic. Density functional theory (DFT) calculation shows that the polarity greatly reduces the excitation energy from the state without net spin (singlet) to the state with net spin (triplet), making the considerable existence of magnetic moments on the polar groups. We note that the hydrophobic groups in these molecules have no magnetic moments, however, they make the molecules aggregate to amply the magnetic effect of the magnetic moments in the polar groups, so that these magnetic moments can induce the strong paramagnetism. Our observations suggest a recovery of the symmetry with inextricable link between the electricity and magnetism at the interface. The findings leave many imaginations of the role of the magnetic interaction in biological systems as well as other magnetic applications considering that many of those polar materials are biological materials, pharmaceutical materials, chemical raw materials, and even an essential hormone in agricultural production.

cond-mat.soft↗

arXiv2022

Behaviour-Diverse Automatic Penetration Testing: A Curiosity-Driven Multi-Objective Deep Reinforcement Learning Approach

Penetration Testing plays a critical role in evaluating the security of a target network by emulating real active adversaries. Deep Reinforcement Learning (RL) is seen as a promising solution to automating the process of penetration tests by reducing human effort and improving reliability. Existing RL solutions focus on finding a specific attack path to impact the target hosts. However, in reality, a diverse range of attack variations are needed to provide comprehensive assessments of the target network's security level. Hence, the attack agents must consider multiple objectives when penetrating the network. Nevertheless, this challenge is not adequately addressed in the existing literature. To this end, we formulate the automatic penetration testing in the Multi-Objective Reinforcement Learning (MORL) framework and propose a Chebyshev decomposition critic to find diverse adversary strategies that balance different objectives in the penetration test. Additionally, the number of available actions increases with the agent consistently probing the target network, making the training process intractable in many practical situations. Thus, we introduce a coverage-based masking mechanism that reduces attention on previously selected actions to help the agent adapt to future exploration. Experimental evaluation on a range of scenarios demonstrates the superiority of our proposed approach when compared to adapted algorithms in terms of multi-objective learning and performance efficiency.

cs.LG↗

arXiv2021

Super strong paramagnetism induced by polar functional groups and water

We experimentally demonstrate that some commonly used materials such as cellulose acetate and chitin which are traditionally considered to be non-magnetic show super strong paramagnetism in aqueous solutions under ambient conditions when they are agglomerated by nanoparticles. Theoretical computations show that strongly polar functional groups can reduce the potential barrier for a singlet-triplet interconversion with the help of surrounding water, inducing the magnetic moments. These magnetic moments distributed on the surfaces of the nanoparticles, which make a large number of magnetic moments gather in a very small space, greatly enhance the alignment of the moments along and amplify the effect of the external magnetic field, resulting in the super strong paramagnetism. Our findings suggest that the polar functional group may always induce paramagnetism and the magnetic effect may be universal as the electric effects since the polarization is very common in materials.

cond-mat.soft↗

arXiv2018

Robust Wireless Body Area Networks Coexistence: A Game Theoretic Approach to Time-Division MAC

The enabling of wireless body area networks (WBANs) coexistence by radio interference mitigation is very important due to a rapid growth in potential users, and a lack of a central coordinator among WBANs that are closely located. In this paper, we propose a TDMA based MAC layer Scheme, with a back-off mechanism that reduces packet collision probability; and estimate performance using a Markov chain model. Based on the MAC layer scheme, a novel non-cooperative game is proposed to jointly adjust sensor node's transmit power and rate. In comparison with the state-of-art, simulation that includes empirical data shows that the proposed approach leads to higher throughput and longer node lifespan as WBAN wearers dynamically move into each other's vicinity. Moreover, by adaptively tuning contention windows size an alternative game is developed, which significantly reduces the latency. Both proposed games provide robust transmission under strong inter-WBAN interferences, but are demonstrated to be applicable to different scenarios. The uniqueness and existence of Nash Equilibrium (NE), as well as close-to-optimum social efficiency, is also proven for both games.

eess.SP↗

arXiv2017

NaCl crystal from salt solution with far below saturated concentration under ambient condition

Under ambient conditions, we directly observed NaCl crystals experimentally in the rGO membranes soaked in the salt solution with concentration below and far below the saturated concentration. Moreover, in most probability, the NaCl crystals show stoichiometries behavior. We attribute this unexpected crystallization to the cation-π interactions between the ions and the aromatic rings of the rGO.

physics.chem-ph↗