arXiv subjects

Lei Chen

Publications and source records attributed to Lei Chen.

At least 235 records · Page 13Linked to original sources

arXiv2025

Quantum gyroscope based on the cavity magnomechanical system

High-precision rotational angle measurement in noise-prone environments holds critical impor tance in aerospace engineering, military navigation, and related domains. In this paper, we propose a quantum gyroscope scheme based on a cavity magnomechanical system, which enables high precision rotation angle detection by harnessing hybrid light-magnon interactions. Central to this framework is the employment of a two-mode squeezed coherent state, generated via parametric coupling of dual quantized optical fields with collective spin excitations (magnons), serving as the quantum metrological probe. We demonstrate that this scheme can significantly reduce quantum noise to levels far below the shot-noise limit. Furthermore, in the non-Markovian case, the per formance of the quantum gyroscope in a dissipative environment does not deteriorate over time, provided that the environmental spectral density satisfies certain conditions. These findings provide critical insights for advancing miniaturized quantum gyroscopes with sub-microradian precision, addressing long-standing challenges in inertial navigation systems under strong ambient noise.

quant-ph↗

arXiv2025

Towards Large-scale Generative Ranking

Generative recommendation has recently emerged as a promising paradigm in information retrieval. However, generative ranking systems are still understudied, particularly with respect to their effectiveness and feasibility in large-scale industrial settings. This paper investigates this topic at the ranking stage of Xiaohongshu's Explore Feed, a recommender system that serves hundreds of millions of users. Specifically, we first examine how generative ranking outperforms current industrial recommenders. Through theoretical and empirical analyses, we find that the primary improvement in effectiveness stems from the generative architecture, rather than the training paradigm. To facilitate efficient deployment of generative ranking, we introduce GenRank, a novel generative architecture for ranking. We validate the effectiveness and efficiency of our solution through online A/B experiments. The results show that GenRank achieves significant improvements in user satisfaction with nearly equivalent computational resources compared to the existing production system.

cs.IR↗

arXiv2025

A machine learning model for skillful climate system prediction

Climate system models (CSMs), through integrating cross-sphere interactions among the atmosphere, ocean, land, and cryosphere, have emerged as pivotal tools for deciphering climate dynamics and improving forecasting capabilities. Recent breakthroughs in artificial intelligence (AI)-driven meteorological modeling have demonstrated remarkable success in single-sphere systems and partially spheres coupled systems. However, the development of a fully coupled AI-based climate system model encompassing atmosphere-ocean-land-sea ice interactions has remained an unresolved challenge. This paper introduces FengShun-CSM, an AI-based CSM model that provides 60-day global daily forecasts for 29 critical variables across atmospheric, oceanic, terrestrial, and cryospheric domains. The model significantly outperforms the European Centre for Medium-Range Weather Forecasts (ECMWF) subseasonal-to-seasonal (S2S) model in predicting most variables, particularly precipitation, land surface, and oceanic components. This enhanced capability is primarily attributed to its improved representation of intra-seasonal variability modes, most notably the Madden-Julian Oscillation (MJO). Remarkably, FengShun-CSM exhibits substantial potential in predicting subseasonal extreme events. Such breakthroughs will advance its applications in meteorological disaster mitigation, marine ecosystem conservation, and agricultural productivity enhancement. Furthermore, it validates the feasibility of developing AI-powered CSMs through machine learning technologies, establishing a transformative paradigm for next-generation Earth system modeling.

cs.LG↗

arXiv2025

Emergent heavy-fermion physics in a new family of topological insulators RAsS (R = Y, La, and Sm)

Realizing topological phases in strongly correlated materials has become a major impetus in condensed matter physics. Although many compounds are now classified as topological insulators, $f$-electron systems (with their strong electron correlations) provide an especially fertile platform for emergent heavy-fermion phenomena driven by the interplay of topology and many-body effects. In this study, we examine the crystalline topology of a new RAsS series (R = Y, La, Sm), revealing a structural variant from previous reports. We demonstrate that YAsS and SmAsS host hourglass fermions protected by glide symmetry. SmAsS notably exhibits a strong effective-mass enhancement, placing it alongside SmB${}_6$ and YbB${}_{12}$ as a material that couples topological surface states with emergent Kondo physics, yet distinguished by its crystalline symmetry constraints and $f$-$p$ orbital hybridization. To capture these features, we construct a minimal model incorporating $f$-electron degrees of freedom, which reproduces the observed topological properties and predicts that the surface states survive in the correlated regime, albeit shifted in energy. Our work thus introduces a new family of correlated topological materials and forecasts the robustness of their surface states under Kondo correlations.

cond-mat.mtrl-sci↗

arXiv2025

InstaRevive: One-Step Image Enhancement via Dynamic Score Matching

Image enhancement finds wide-ranging applications in real-world scenarios due to complex environments and the inherent limitations of imaging devices. Recent diffusion-based methods yield promising outcomes but necessitate prolonged and computationally intensive iterative sampling. In response, we propose InstaRevive, a straightforward yet powerful image enhancement framework that employs score-based diffusion distillation to harness potent generative capability and minimize the sampling steps. To fully exploit the potential of the pre-trained diffusion model, we devise a practical and effective diffusion distillation pipeline using dynamic control to address inaccuracies in updating direction during score matching. Our control strategy enables a dynamic diffusing scope, facilitating precise learning of denoising trajectories within the diffusion model and ensuring accurate distribution matching gradients during training. Additionally, to enrich guidance for the generative power, we incorporate textual prompts via image captioning as auxiliary conditions, fostering further exploration of the diffusion model. Extensive experiments substantiate the efficacy of our framework across a diverse array of challenging tasks and datasets, unveiling the compelling efficacy and efficiency of InstaRevive in delivering high-quality and visually appealing results. Code is available at https://github.com/EternalEvan/InstaRevive.

cs.CV↗

arXiv2025

Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving

Large language model (LLM) inference serving systems are essential to various LLM-based applications. As demand for LLM services continues to grow, scaling these systems to handle high request rates while meeting latency Service-Level Objectives (SLOs), referred to as effective throughput, becomes critical. However, existing systems often struggle to improve effective throughput, primarily due to a significant decline in Time To First Token (TTFT) SLO attainment. We identify two major causes of this bottleneck: (1) memory-intensive KV cache that limits batch size expansion under GPU memory constraints, and (2) rigid batch composition enforced by the default First-Come-First-Serve scheduling policy. In this paper, we introduce Apt-Serve, a scalable framework designed to enhance effective throughput in LLM inference serving. Apt-Serve features a new hybrid cache scheme that combines KV cache with a memory-efficient hidden cache for reusable input hidden state vectors, allowing large batch sizes and improving request concurrency. Based on the hybrid cache, Apt-Serve employs an adaptive runtime scheduling mechanism that dynamically optimizes batch composition. We formally define the adaptive scheduling optimization problem and propose an efficient algorithm with theoretical guarantees. Extensive evaluations on three real-world datasets and LLMs ranging from 13B to 66B parameters demonstrate that Apt-Serve achieves up to 8.8x improvement in effective throughput compared to the state-of-the-art inference serving systems.

cs.LG↗

arXiv2025

Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions

Fine-tuning has emerged as a critical process in leveraging Large Language Models (LLMs) for specific downstream tasks, enabling these models to achieve state-of-the-art performance across various domains. However, the fine-tuning process often involves sensitive datasets, introducing privacy risks that exploit the unique characteristics of this stage. In this paper, we provide a comprehensive survey of privacy challenges associated with fine-tuning LLMs, highlighting vulnerabilities to various privacy attacks, including membership inference, data extraction, and backdoor attacks. We further review defense mechanisms designed to mitigate privacy risks in the fine-tuning phase, such as differential privacy, federated learning, and knowledge unlearning, discussing their effectiveness and limitations in addressing privacy risks and maintaining model utility. By identifying key gaps in existing research, we highlight challenges and propose directions to advance the development of privacy-preserving methods for fine-tuning LLMs, promoting their responsible use in diverse applications.

cs.AI↗

arXiv2025

Learning Towards Emergence: Paving the Way to Induce Emergence by Inhibiting Monosemantic Neurons on Pre-trained Models

Emergence, the phenomenon of a rapid performance increase once the model scale reaches a threshold, has achieved widespread attention recently. The literature has observed that monosemantic neurons in neural networks gradually diminish as the model scale increases. Subsequently, Learning From Emergence is proposed to actively inhibit monosemantic neurons in relatively small neural networks (e.g., BERT and Swin-Transformer) for promoting model performance with fine-tuning. However, to ultimately achieve emergence, it is demanding to support the monosemantic neuron inhibition in the pretraining phase of large-scale models. Thus, this work further pushes the boundary of this research direction to be Learning Towards Emergence (L2E) and enables the training and validating of the impact of inhibiting monosemantic neurons on larger pre-trained neural networks (e.g., Pythia-70M, 410M, and 2.8B). More specifically, to bridge the gap in current research, we first conduct experiments on models of various scales (up to 6.9B) to validate the monosemantic ideas. Then, we present a novel method L2E to address the inefficient monosemantic neuron retrieval and ineffective monosemantic neuron inhibition when existing methods are applied in the pretraining phase of large-scale models. It employs an adjustable thresholding technique for efficient neuron retrieval, incorporates a False Killing Rate metric to assess inhibition effects, and proposes a regularization-style inhibition approach, which addresses the limitations of previous approaches in both efficiency and effectiveness. Experimental results demonstrate the effectiveness of L2E's monosemantic neuron inhibition and its efficiency in implementation with large-scale models.

cs.ET↗

arXiv2025

FuXi-RTM: A Physics-Guided Prediction Framework with Radiative Transfer Modeling

Similar to conventional video generation, current deep learning-based weather prediction frameworks often lack explicit physical constraints, leading to unphysical outputs that limit their reliability for operational forecasting. Among various physical processes requiring proper representation, radiation plays a fundamental role as it drives Earth's weather and climate systems. However, accurate simulation of radiative transfer processes remains challenging for traditional numerical weather prediction (NWP) models due to their inherent complexity and high computational costs. Here, we propose FuXi-RTM, a hybrid physics-guided deep learning framework designed to enhance weather forecast accuracy while enforcing physical consistency. FuXi-RTM integrates a primary forecasting model (FuXi) with a fixed deep learning-based radiative transfer model (DLRTM) surrogate that efficiently replaces conventional radiation parameterization schemes. This represents the first deep learning-based weather forecasting framework to explicitly incorporate physical process modeling. Evaluated over a comprehensive 5-year dataset, FuXi-RTM outperforms its unconstrained counterpart in 88.51% of 3320 variable and lead time combinations, with improvements in radiative flux predictions. By incorporating additional physical processes, FuXi-RTM paves the way for next-generation weather forecasting systems that are both accurate and physically consistent.

physics.ao-ph↗

arXiv2025

Improving the End-to-End Efficiency of Offline Inference for Multi-LLM Applications Based on Sampling and Simulation

As large language models (LLMs) have shown great success in many tasks, they are used in various applications. While a lot of works have focused on the efficiency of single-LLM application (e.g., offloading, request scheduling, parallelism strategy selection), multi-LLM applications receive less attention, particularly in offline inference scenarios. In this work, we aim to improve the offline end-to-end inference efficiency of multi-LLM applications in the single-node multi-GPU environment. The problem involves two key decisions: (1) determining which LLMs to run concurrently each time (we may not run all the models at the same time), and (2) selecting a parallelism strategy to use for each LLM. This problem is NP-hard. Naive solutions may not work well because the running time for a model to complete a set of requests depends on the request workload and the selected parallelism strategy, and they lack an accurate model of the running time. As the LLM output lengths are unknown before running, to estimate the model running time, we propose a sampling-then-simulation method which first estimates the output lengths by sampling from an empirical cumulative function we obtained from a large dataset in advance, and then simulates the LLM inference process accordingly. Based on the simulation, we estimate the per-iteration latencys to get the total latency. A greedy method is proposed to optimize the scheduling of the LLMs in the application across the GPUs. We then propose a framework SamuLLM which contains two phases: planning, which calls the greedy method for an application and running, which runs the application and dynamically adjust the model scheduling based on the runtime information. Experiments on 3 applications and a mixed application show that SamuLLM can achieve 1.0-2.4$\times$ end-to-end speedups compared to the competitors.

cs.DC↗

arXiv2025

PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing

While scaling laws have been continuously validated in large language models (LLMs) with increasing model parameters, the inherent tension between the inference demands of LLMs and the limited resources of edge devices poses a critical challenge to the development of edge intelligence. Recently, numerous small language models have emerged, aiming to distill the capabilities of LLMs into smaller footprints. However, these models often retain the fundamental architectural principles of their larger counterparts, still imposing considerable strain on the storage and bandwidth capacities of edge devices. In this paper, we introduce the PLM, a Peripheral Language Model, developed through a co-design process that jointly optimizes model architecture and edge system constraints. The PLM utilizes a Multi-head Latent Attention mechanism and employs the squared ReLU activation function to encourage sparsity, thereby reducing peak memory footprint during inference. During training, we collect and reorganize open-source datasets, implement a multi-phase training strategy, and empirically investigate the Warmup-Stable-Decay-Constant (WSDC) learning rate scheduler. Additionally, we incorporate Reinforcement Learning from Human Feedback (RLHF) by adopting the ARIES preference learning approach. Following a two-phase SFT process, this method yields performance gains of 2% in general tasks, 9% in the GSM8K task, and 11% in coding tasks. In addition to its novel architecture, evaluation results demonstrate that PLM outperforms existing small language models trained on publicly available data while maintaining the lowest number of activated parameters. Furthermore, deployment across various edge devices, including consumer-grade GPUs, mobile phones, and Raspberry Pis, validates PLM's suitability for peripheral applications. The PLM series models are publicly available at https://github.com/plm-team/PLM.

cs.CL↗

arXiv2025

A Tale of Single-channel Electroencephalogram: Devices, Datasets, Signal Processing, Applications, and Future Directions

Single-channel electroencephalogram (EEG) is a cost-effective, comfortable, and non-invasive method for monitoring brain activity, widely adopted by researchers, consumers, and clinicians. The increasing number and proportion of articles on single-channel EEG underscore its growing potential. This paper provides a comprehensive review of single-channel EEG, focusing on development trends, devices, datasets, signal processing methods, recent applications, and future directions. Definitions of bipolar and unipolar configurations in single-channel EEG are clarified to guide future advancements. Applications mainly span sleep staging, emotion recognition, educational research, and clinical diagnosis. Ongoing advancements of single-channel EEG in AI-based EEG generation techniques suggest potential parity or superiority over multichannel EEG performance.

cs.CV↗

arXiv2025

E-SAM: Training-Free Segment Every Entity Model

Entity Segmentation (ES) aims at identifying and segmenting distinct entities within an image without the need for predefined class labels. This characteristic makes ES well-suited to open-world applications with adaptation to diverse and dynamically changing environments, where new and previously unseen entities may appear frequently. Existing ES methods either require large annotated datasets or high training costs, limiting their scalability and adaptability. Recently, the Segment Anything Model (SAM), especially in its Automatic Mask Generation (AMG) mode, has shown potential for holistic image segmentation. However, it struggles with over-segmentation and under-segmentation, making it less effective for ES. In this paper, we introduce E-SAM, a novel training-free framework that exhibits exceptional ES capability. Specifically, we first propose Multi-level Mask Generation (MMG) that hierarchically processes SAM's AMG outputs to generate reliable object-level masks while preserving fine details at other levels. Entity-level Mask Refinement (EMR) then refines these object-level masks into accurate entity-level masks. That is, it separates overlapping masks to address the redundancy issues inherent in SAM's outputs and merges similar masks by evaluating entity-level consistency. Lastly, Under-Segmentation Refinement (USR) addresses under-segmentation by generating additional high-confidence masks fused with EMR outputs to produce the final ES map. These three modules are seamlessly optimized to achieve the best ES without additional training overhead. Extensive experiments demonstrate that E-SAM achieves state-of-the-art performance compared to prior ES methods, demonstrating a significant improvement by +30.1 on benchmark metrics.

cs.CV↗

arXiv2025

Resolving the Kagome Origin of the Strange Metallicity in Ni$_3$In

Strong correlations promote singular properties such as strange metallicity, which shows considerable commonality across quantum materials platforms. Understanding the mechanism for such emerging universality is an outstanding challenge, given that the underlying degrees of freedom can be complex and varied. Progress may be made in flat band systems, especially kagome and other frustrated-lattice metals with active flat bands. These systems show strange metal behavior that bears a striking resemblance to what happens in heavy-fermion metals. Here, in scanning tunneling spectroscopy of kagome metal Ni$_3$In, we find a zero-bias peak-dip structure whose variation with magnetic field and temperature tracks the evolution of the strange metal properties. We identify the origin of the peak as compact molecular orbitals formed by destructive interference over the kagome sites, resulting in emergent $f$-shell-like localized moments. Using quasi-particle interference, we visualize their interaction with the Dirac light bands. We thus unveil the essential microscopic ingredients of the $d$-electron-based kagome metals that, while distinct from the atomic orbitals of the $f$-electron-based heavy fermion materials, are responsible for a shared phenomenology between the two types of systems. Our findings provide a new window to uncover and interconnect the essential and yet diverse microscopic building blocks in disparate families of quantum materials that drive a convergence towards a universal understanding in the regime of amplified quantum fluctuations.

cond-mat.str-el↗

arXiv2025

Correlated flat-band physics in a bilayer kagome metal based on compact molecular orbitals

Flat bands, when located close to the Fermi energy, can considerably enhance the influence of electron correlations on the low energy physics in kagome and other frustrated-lattice metals. A major challenge in describing the interaction effects in such bulk materials is that the flat band is often intermixed with a large number of other bands. Here we show that the recently introduced notion of compact molecular orbitals (CMOs) enable a path forward in describing the dominant effect of the Coulomb interactions in spite of the complexity of the bandstructure. Our materials-based analysis allows for the understanding of the scanning-tunneling-microscopy experiment [J. C. Souza et al., preprint (2024)] of the bilayer kagome metal Ni$_3$In in terms of the CMO notion. From the resulting CMO, an effective Anderson lattice model can be set up. This CMO-based approach enables the calculation of correlation effects that is difficult to do based on the atomic orbitals. Furthermore, it suggests an enriched phase diagram for the strange metal physics of the kagome metal, which can be tested by future experiments. We discuss the implications of our results for the general correlation physics of flat band systems and beyond.

cond-mat.str-el↗

arXiv2025

Quantum critical metals and loss of quasiparticles

Strange metals develop near quantum critical points in a variety of strongly correlated systems. Some of the issues that are central to the field include how the quantum-critical state loses quasiparticles, how it drives superconductivity, and to what extent the strange-metal physics in different classes of correlated systems are interconnected. In this Review, we survey some of these issues from the vantage point of heavy fermion metals. We will describe the notion of Kondo destruction and how it leads to several crucial effects. These include a transformation of the Fermi surface from large to small when the system is tuned across the quantum-critical point, a loss of quasiparticles everywhere on the Fermi surface when it is perched at the quantum-critical point, and a dynamical Planckian scaling in various physical properties including charge responses. We close with a discussion about the connections between the strange-metal physics in heavy fermion metals and its counterparts in the cuprates and other correlated materials.

cond-mat.str-el↗

arXiv2025

Metallic quantum criticality enabled by flat bands in a kagome lattice

Strange metals arise in a variety of platforms for strongly correlated electrons, ranging from the cuprates, heavy fermions to flat band systems. Motivated by recent experiments in kagome metals, we study a Hubbard model on a kagome lattice whose noninteracting limit contains flat bands. A Kondo lattice description is constructed, in which the correlation effects are captured by symmetry preserving and exponentially localized molecular orbitals. These compact molecular orbitals represent the local degrees of freedom that emerge from topological flat bands. We identify a quantum critical point at which quasiparticles are lost and strange metallicity emerges. Our theoretical work opens up a new route for realizing beyond-Landau quantum criticality, as well as the associated strange metallicity and emergent quantum phases.

cond-mat.str-el↗

arXiv2025

KAA: Kolmogorov-Arnold Attention for Enhancing Attentive Graph Neural Networks

Graph neural networks (GNNs) with attention mechanisms, often referred to as attentive GNNs, have emerged as a prominent paradigm in advanced GNN models in recent years. However, our understanding of the critical process of scoring neighbor nodes remains limited, leading to the underperformance of many existing attentive GNNs. In this paper, we unify the scoring functions of current attentive GNNs and propose Kolmogorov-Arnold Attention (KAA), which integrates the Kolmogorov-Arnold Network (KAN) architecture into the scoring process. KAA enhances the performance of scoring functions across the board and can be applied to nearly all existing attentive GNNs. To compare the expressive power of KAA with other scoring functions, we introduce Maximum Ranking Distance (MRD) to quantitatively estimate their upper bounds in ranking errors for node importance. Our analysis reveals that, under limited parameters and constraints on width and depth, both linear transformation-based and MLP-based scoring functions exhibit finite expressive power. In contrast, our proposed KAA, even with a single-layer KAN parameterized by zero-order B-spline functions, demonstrates nearly infinite expressive power. Extensive experiments on both node-level and graph-level tasks using various backbone models show that KAA-enhanced scoring functions consistently outperform their original counterparts, achieving performance improvements of over 20% in some cases.

cs.LG↗