arXiv subjects

Lei Chen

Publications and source records attributed to Lei Chen.

At least 253 records · Page 14Linked to original sources

arXiv2025

OpenRAG: Optimizing RAG End-to-End via In-Context Retrieval Learning

In this paper, we analyze and empirically show that the learned relevance for conventional information retrieval (IR) scenarios may be inconsistent in retrieval-augmented generation (RAG) scenarios. To bridge this gap, we introduce OpenRAG, a RAG framework that is optimized end-to-end by tuning the retriever to capture in-context relevance, enabling adaptation to the diverse and evolving needs. Extensive experiments across a wide range of tasks demonstrate that OpenRAG, by tuning a retriever end-to-end, leads to a consistent improvement of 4.0% over the original retriever, consistently outperforming existing state-of-the-art retrievers by 2.1%. Additionally, our results indicate that for some tasks, an end-to-end tuned 0.2B retriever can achieve improvements that surpass those of RAG-oriented or instruction-tuned 8B large language models (LLMs), highlighting the cost-effectiveness of our approach in enhancing RAG systems.

cs.CL↗

arXiv2025

Amplified multipartite entanglement witnessed in a quantum critical metal

Strong correlations in matter promote a landscape of quantum phases and associated quantum critical points. For metallic systems, there is increasing recognition that the quantum criticality goes beyond the Landau framework and, thus, novel means are needed to characterize the quantum critical fluid. Here we do so by studying an entanglement quantity, the quantum Fisher information, in a strange metal system, focusing on the exemplary case of an Anderson/Kondo lattice model near its Kondo destruction quantum critical point. The spin quantum Fisher information peaks at the quantum critical point and indicates a strongly entangled ground state. Our results are supported by the quantum Fisher information extracted from inelastic neutron scattering measurements in heavy fermion metals. Our work elucidates the loss of quasiparticles in strange metals, opens a quantum information avenue to advance the understanding of metallic quantum criticality in a broad range of strongly correlated systems, and points to a novel regime of quantum matter to realize amplified entanglement.

cond-mat.str-el↗

arXiv2025

E2ED^2:Direct Mapping from Noise to Data for Enhanced Diffusion Models

Diffusion models have established themselves as the de facto primary paradigm in visual generative modeling, revolutionizing the field through remarkable success across various diverse applications ranging from high-quality image synthesis to temporal aware video generation. Despite these advancements, three fundamental limitations persist, including 1) discrepancy between training and inference processes, 2) progressive information leakage throughout the noise corruption procedures, and 3) inherent constraints preventing effective integration of modern optimization criteria like perceptual and adversarial loss. To mitigate these critical challenges, we in this paper present a novel end-to-end learning paradigm that establishes direct optimization from the final generated samples to initial noises. Our proposed End-to-End Differentiable Diffusion, dubbed E2ED^2, introduces several key improvements: it eliminates the sequential training-sampling mismatch and intermediate information leakage via conceptualizing training as a direct transformation from isotropic Gaussian noise to the target data distribution. Additionally, such training framework enables seamless incorporation of adversarial and perceptual losses into the core optimization objective. Comprehensive evaluation across standard benchmarks including COCO30K and HW30K reveals that our method achieves substantial performance gains in terms of Fréchet Inception Distance (FID) and CLIP score, even with fewer sampling steps (less than 4). Our findings highlight that the end-to-end mechanism might pave the way for more robust and efficient solutions, \emph{i.e.,} combining diffusion stability with GAN-like discriminative optimization in an end-to-end manner.

cs.CV↗

arXiv2025

Rule-Based Conflict-Free Decision Framework in Swarm Confrontation

Traditional rule-based decision-making methods with interpretable advantage, such as finite state machine, suffer from the jitter or deadlock(JoD) problems in extremely dynamic scenarios. To realize agent swarm confrontation, decision conflicts causing many JoD problems are a key issue to be solved. Here, we propose a novel decision-making framework that integrates probabilistic finite state machine, deep convolutional networks, and reinforcement learning to implement interpretable intelligence into agents. Our framework overcomes state machine instability and JoD problems, ensuring reliable and adaptable decisions in swarm confrontation. The proposed approach demonstrates effective performance via enhanced human-like cooperation and competitive strategies in the rigorous evaluation of real experiments, outperforming other methods.

cs.AI↗

arXiv2025

Superconductivity in twisted WSe$_2$ from topology-induced quantum fluctuations

Recently, superconductivity has been observed in twisted WSe$_2$ moiré structures (Xia et al., Nature 2024; Guo et al., Nature 2025). Its transition temperature is high, reaching a few percent of the Fermi temperature scale. Here, we advance a mechanism for superconductivity based on the notion that electronic topology enables quantum fluctuations in a suitable regime of intermediate correlations. In this regime, the Coulomb interaction requires that an active topological flat band and nearby wider bands are considered together. Compact molecular orbitals arise, which give rise to quantum fluctuations through topology-dictated hybridization with the other molecular orbitals. The hybridization competes with the active flat band's natural tendency towards static electronic ordering, thereby weakening the latter; we link this effect with certain salient observations by experiments. Furthermore, the competition yields a quantum critical regime where quasiparticles are lost. The corresponding quantum critical fluctuations drive superconductivity. Broader implications and new connections among correlated materials platforms are discussed.

cond-mat.str-el↗

arXiv2025

Semi-Parametric Retrieval via Binary Bag-of-Tokens Index

Information retrieval has transitioned from standalone systems into essential components across broader applications, with indexing efficiency, cost-effectiveness, and freshness becoming increasingly critical yet often overlooked. In this paper, we introduce SemI-parametric Disentangled Retrieval (SiDR), a bi-encoder retrieval framework that decouples retrieval index from neural parameters to enable efficient, low-cost, and parameter-agnostic indexing for emerging use cases. Specifically, in addition to using embeddings as indexes like existing neural retrieval methods, SiDR supports a non-parametric tokenization index for search, achieving BM25-like indexing complexity with significantly better effectiveness. Our comprehensive evaluation across 16 retrieval benchmarks demonstrates that SiDR outperforms both neural and term-based retrieval baselines under the same indexing workload: (i) When using an embedding-based index, SiDR exceeds the performance of conventional neural retrievers while maintaining similar training complexity; (ii) When using a tokenization-based index, SiDR drastically reduces indexing cost and time, matching the complexity of traditional term-based retrieval, while consistently outperforming BM25 on all in-domain datasets; (iii) Additionally, we introduce a late parametric mechanism that matches BM25 index preparation time while outperforming other neural retrieval baselines in effectiveness.

cs.CL↗

arXiv2025

Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers

Large language models have been successful at tasks involving basic forms of in-context reasoning, such as generating coherent language, as well as storing vast amounts of knowledge. At the core of the Transformer architecture behind such models are feed-forward and attention layers, which are often associated to knowledge and reasoning, respectively. In this paper, we study this distinction empirically and theoretically in a controlled synthetic setting where certain next-token predictions involve both distributional and in-context information. We find that feed-forward layers tend to learn simple distributional associations such as bigrams, while attention layers focus on in-context reasoning. Our theoretical analysis identifies the noise in the gradients as a key factor behind this discrepancy. Finally, we illustrate how similar disparities emerge in pre-trained models through ablations on the Pythia model family on simple reasoning tasks.

cs.LG↗

arXiv2025

Patterning Silver Nanowire Network via the Gibbs-Thomson Effect

As transparent electrodes, patterned silver nanowire (AgNW) networks suffer from noticeable pattern visibility, which is an unsettled issue for practical applications such as display. Here, we introduce a Gibbs-Thomson effect (GTE)-based patterning method to effectively reduce pattern visibility. Unlike conventional top-down and bottom-up strategies that rely on selective etching, removal, or deposition of AgNWs, our approach focuses on fragmenting nanowires primarily at the junctions through the GTE. This is realized by modifying AgNWs with a compound of diphenyliodonium nitrate and silver nitrate, which aggregates into nanoparticles at the junctions of AgNWs. These nanoparticles can boost the fragmentation of nanowires at the junctions under an ultralow temperature (75°C), allow pattern transfer through a photolithographic masking operation, and enhance plasmonic welding during UV exposure. The resultant patterned electrodes have trivial differences in transmittance (ΔT = 1.4%) and haze (ΔH = 0.3%) between conductive and insulative regions, with high-resolution patterning size down to 10 μm. To demonstrate the practicality of this novel method, we constructed a highly transparent, optoelectrical interactive tactile e-skin using the patterned AgNW electrodes.

eess.SY↗

arXiv2025

A Selective Learning Method for Temporal Graph Continual Learning

Node classification is a key task in temporal graph learning (TGL). Real-life temporal graphs often introduce new node classes over time, but existing TGL methods assume a fixed set of classes. This assumption brings limitations, as updating models with full data is costly, while focusing only on new classes results in forgetting old ones. Graph continual learning (GCL) methods mitigate forgetting using old-class subsets but fail to account for their evolution. We define this novel problem as temporal graph continual learning (TGCL), which focuses on efficiently maintaining up-to-date knowledge of old classes. To tackle TGCL, we propose a selective learning framework that substitutes the old-class data with its subsets, Learning Towards the Future (LTF). We derive an upper bound on the error caused by such replacement and transform it into objectives for selecting and learning subsets that minimize classification error while preserving the distribution of the full old-class data. Experiments on three real-world datasets validate the effectiveness of LTF on TGCL.

cs.LG↗

arXiv2025

Extended Impedance Modal Analysis of Internal Dynamics in Grid-Following Inverters

As the penetration of Grid-Following inverters (GFL) in power systems continues to increase, the dynamic characteristics of power systems undergo significant transformations. Recently, modal analysis based on the impedance model (MAI) has been utilized to evaluate the interaction between GFLs and the power grid through impedance/admittance participation factors (PF). However, MAI relies on the impedance model that characterizes the input-output behavior of the ports, treating the GFL as a single integrated entity. This approach limits its ability to reveal the complex dynamic coupling mechanism between different control loops inside the GFL. In this paper, we propose an extended impedance modal analysis (EMAI) method. Firstly, the equivalent dynamics of the GFL are decomposed into two components: the synchronous dynamic dominated by the phase-locked loop (PLL) and the electromagnetic dynamic dominated by the current control loop (CCL). The PF and participation ratios (PR) of these two dynamic components are then calculated to identify the dominant dynamics of the system. Building on this, we introduce the explicit parameter participation factors (PPF) to further pinpoint the key parameters of the dominant control loop, which provides a way for enhancing system stability. Finally, the effectiveness of the proposed method is validated through the simulation of the modified 14-bus and 68-bus systems. The EMAI method enables the analysis of the dynamic characteristics of each control loop in the GFL based on the impedance model. It can effectively identify the critical control loops influencing system stability without requiring the construction of a full state-space model, demonstrating its broad applicability and value.

eess.SY↗

arXiv2025

Logic Synthesis Optimization with Predictive Self-Supervision via Causal Transformers

Contemporary hardware design benefits from the abstraction provided by high-level logic gates, streamlining the implementation of logic circuits. Logic Synthesis Optimization (LSO) operates at one level of abstraction within the Electronic Design Automation (EDA) workflow, targeting improvements in logic circuits with respect to performance metrics such as size and speed in the final layout. Recent trends in the field show a growing interest in leveraging Machine Learning (ML) for EDA, notably through ML-guided logic synthesis utilizing policy-based Reinforcement Learning (RL) methods.Despite these advancements, existing models face challenges such as overfitting and limited generalization, attributed to constrained public circuits and the expressiveness limitations of graph encoders. To address these hurdles, and tackle data scarcity issues, we introduce LSOformer, a novel approach harnessing Autoregressive transformer models and predictive SSL to predict the trajectory of Quality of Results (QoR). LSOformer integrates cross-attention modules to merge insights from circuit graphs and optimization sequences, thereby enhancing prediction accuracy for QoR metrics. Experimental studies validate the effectiveness of LSOformer, showcasing its superior performance over baseline architectures in QoR prediction tasks, where it achieves improvements of 5.74%, 4.35%, and 17.06% on the EPFL, OABCD, and proprietary circuits datasets, respectively, in inductive setup.

cs.AI↗

arXiv2025

Activation-aware Probe-Query: Effective Key-Value Retrieval for Long-Context LLMs Inference

Recent advances in large language models (LLMs) have showcased exceptional performance in long-context tasks, while facing significant inference efficiency challenges with limited GPU memory. Existing solutions first proposed the sliding-window approach to accumulate a set of historical \textbf{key-value} (KV) pairs for reuse, then further improvements selectively retain its subsets at each step. However, due to the sparse attention distribution across a long context, it is hard to identify and recall relevant KV pairs, as the attention is distracted by massive candidate pairs. Additionally, we found it promising to select representative tokens as probe-Query in each sliding window to effectively represent the entire context, which is an approach overlooked by existing methods. Thus, we propose \textbf{ActQKV}, a training-free, \textbf{Act}ivation-aware approach that dynamically determines probe-\textbf{Q}uery and leverages it to retrieve the relevant \textbf{KV} pairs for inference. Specifically, ActQKV monitors a token-level indicator, Activation Bias, within each context window, enabling the proper construction of probe-Query for retrieval at pre-filling stage. To accurately recall the relevant KV pairs and minimize the irrelevant ones, we design a dynamic KV cut-off mechanism guided by information density across layers at the decoding stage. Experiments on the Long-Bench and $\infty$ Benchmarks demonstrate its state-of-the-art performance with competitive inference quality and resource efficiency.

cs.CL↗

arXiv2025

Cardinality Estimation on Hyper-relational Knowledge Graphs

Cardinality Estimation (CE) for query is to estimate the number of results without execution, which is an effective index in query optimization. Recently, CE for queries over knowlege graph (KGs) with triple facts has achieved great success. To more precisely represent facts, current researchers propose hyper-relational KGs (HKGs) to represent a triple fact with qualifiers providing additional context to the fact. However, existing CE methods, such as sampling and summary methods over KGs, perform unsatisfactorily on HKGs due to the complexity of qualifiers. Learning-based CE methods do not utilize qualifier information to learn query representation accurately, leading to poor performance. Also, there is only one limited CE benchmark for HKG query, which is not comprehensive and only covers limited patterns. The lack of querysets over HKG also becomes a bottleneck to comprehensively investigate CE problems on HKGs. In this work, we first construct diverse and unbiased hyper-relational querysets over three popular HKGs for investigating CE. Besides, we also propose a novel qualifier-aware graph neural network (GNN) model that effectively incorporates qualifier information and adaptively combines outputs from multiple GNN layers, to accurately predict the cardinality. Our experiments demonstrate that our model outperforms all state-of-the-art CE methods over three benchmarks on popular HKGs.

cs.LG↗

arXiv2025

Evidence for an accretion bridge in the DX Cha circumbinary system from VLTI/MATISSE observations

DX Cha (HD 104237) is a spectroscopic binary consisting of a Herbig A7.5Ve-A8Ve primary star and a K3-type companion. Here we report on new $3.55$ micrometer interferometric observations of this source with the Multi Aperture Mid-Infrared Spectroscopic Experiment (MATISSE) at the Very Large Telescope Interferometer (VLTI). To model the four MATISSE observations obtained between 2020 and 2023, we constructed a time-dependent interferometric model of the system, using the oimodeler software. The model consists of an asymmetric ring and two point sources on a Keplerian orbit. Our best-fit model consists of a circumbinary ring with a diameter of $0.86$ au ($8.1$ mas), featuring a strong azimuthal asymmetry. We found that the position angle of the asymmetry changes tens of degrees between the MATISSE epochs. The ring is relatively narrow, with a full width at half maximum (FWHM) of $\sim$$0.13$ au ($1.23$ mas). The presence of circumstellar dust emission so close to the binary is unexpected, as previous hydrodynamic simulations predicted an inner disk cavity with a diameter of $\sim$$4$ au ($\sim$$37.5$ mas). Thus, we argue that the narrow envelope of material we detected is probably not a gravitationally stable circumbinary ring, but may be part of tidal accretion streamers channeling material from the inner edge of the disk toward the stars.

astro-ph.SR↗

arXiv2025

Orbital-selective correlation effects and superconducting pairing symmetry in a multiorbital $t$-$J$ model for bilayer nickelates

The recent discovery of superconductivity in La$_3$Ni$_2$O$_7$ raises key questions about its mechanism and the nature of pairing symmetry. This system is believed to be described by a bilayer two-orbital Hubbard model. The considerations of orbital-selective Mott correlations motivate a bilayer two-orbital $t$-$J$ model and, accordingly, we study the superconducting pairing in this model. We obtain an overall phase diagram of superconductivity, where the leading channel has either extended $s$-wave or $d_{x^2-y^2}$-wave symmetry. Our analysis highlights how the orbital-selective correlations affect the superconducting pairing via the interlayer exchange couplings and low-energy electronic structure. In particular, we find that the dominant orbital for the pairing may change between $z^2$ and $x^2-y^2$ when the position of the bonding $z^2$ band is varied by tuning either the $c$-axis lattice constant or electron concentration strength. We discuss the implications of these results for the superconductivity in both bulk La$_{3}$Ni$_{2}$O$_{7}$ and its thin film counterpart.

cond-mat.supr-con↗

arXiv2025

When Data Manipulation Meets Attack Goals: An In-depth Survey of Attacks for VLMs

Vision-Language Models (VLMs) have gained considerable prominence in recent years due to their remarkable capability to effectively integrate and process both textual and visual information. This integration has significantly enhanced performance across a diverse spectrum of applications, such as scene perception and robotics. However, the deployment of VLMs has also given rise to critical safety and security concerns, necessitating extensive research to assess the potential vulnerabilities these VLM systems may harbor. In this work, we present an in-depth survey of the attack strategies tailored for VLMs. We categorize these attacks based on their underlying objectives - namely jailbreak, camouflage, and exploitation - while also detailing the various methodologies employed for data manipulation of VLMs. Meanwhile, we outline corresponding defense mechanisms that have been proposed to mitigate these vulnerabilities. By discerning key connections and distinctions among the diverse types of attacks, we propose a compelling taxonomy for VLM attacks. Moreover, we summarize the evaluation metrics that comprehensively describe the characteristics and impact of different attacks on VLMs. Finally, we conclude with a discussion of promising future research directions that could further enhance the robustness and safety of VLMs, emphasizing the importance of ongoing exploration in this critical area of study. To facilitate community engagement, we maintain an up-to-date project page, accessible at: https://github.com/AobtDai/VLM_Attack_Paper_List.

cs.CV↗

arXiv2025

Enhancing Cross-domain Link Prediction via Evolution Process Modeling

This work proposes DyExpert, a dynamic graph model for cross-domain link prediction. It can explicitly model historical evolving processes to learn the evolution pattern of a specific downstream graph and subsequently make pattern-specific link predictions. DyExpert adopts a decode-only transformer and is capable of efficiently parallel training and inference by \textit{conditioned link generation} that integrates both evolution modeling and link prediction. DyExpert is trained by extensive dynamic graphs across diverse domains, comprising 6M dynamic edges. Extensive experiments on eight untrained graphs demonstrate that DyExpert achieves state-of-the-art performance in cross-domain link prediction. Compared to the advanced baseline under the same setting, DyExpert achieves an average of 11.40% improvement Average Precision across eight graphs. More impressive, it surpasses the fully supervised performance of 8 advanced baselines on 6 untrained graphs.

cs.LG↗

arXiv2025

SpaceGNN: Multi-Space Graph Neural Network for Node Anomaly Detection with Extremely Limited Labels

Node Anomaly Detection (NAD) has gained significant attention in the deep learning community due to its diverse applications in real-world scenarios. Existing NAD methods primarily embed graphs within a single Euclidean space, while overlooking the potential of non-Euclidean spaces. Besides, to address the prevalent issue of limited supervision in real NAD tasks, previous methods tend to leverage synthetic data to collect auxiliary information, which is not an effective solution as shown in our experiments. To overcome these challenges, we introduce a novel SpaceGNN model designed for NAD tasks with extremely limited labels. Specifically, we provide deeper insights into a task-relevant framework by empirically analyzing the benefits of different spaces for node representations, based on which, we design a Learnable Space Projection function that effectively encodes nodes into suitable spaces. Besides, we introduce the concept of weighted homogeneity, which we empirically and theoretically validate as an effective coefficient during information propagation. This concept inspires the design of the Distance Aware Propagation module. Furthermore, we propose the Multiple Space Ensemble module, which extracts comprehensive information for NAD under conditions of extremely limited supervision. Our findings indicate that this module is more beneficial than data augmentation techniques for NAD. Extensive experiments conducted on 9 real datasets confirm the superiority of SpaceGNN, which outperforms the best rival by an average of 8.55% in AUC and 4.31% in F1 scores. Our code is available at https://github.com/xydong127/SpaceGNN.

cs.LG↗