arXiv subjects

Zheng Zhang

Publications and source records attributed to Zheng Zhang.

At least 235 records · Page 13Linked to original sources

arXiv2025

Dominant Kitaev interaction and field-induced quantum phase transitions in triangular-lattice KCeSe2

Realizing Kitaev interactions on triangular lattices offers a compelling platform for exploring quantum-spin-liquid physics beyond the conventional honeycomb lattice framework. Here, we investigate the triangular-lattice antiferromagnet KCeSe2, where multiple probes reveal strong magnetic anisotropy suggesting significant Kitaev physics. Through detailed and combined analysis of magnetization, neutron scattering, and thermodynamic experiments, we identify dominant ferromagnetic Kitaev ($K = -1.82$ K) and antiferromagnetic Heisenberg ($J = 1.34$ K) interactions that stabilize a stripe-$yz$ ordered ground state via an order-by-disorder mechanism. Magnetic fields applied along the Kitaev bond direction induce two phase transitions at 1.67 T and 3.8 T, consistent with density matrix renormalization group (DMRG) calculations predictions of a progression from stripe-$yz$ to stripe-canted and spin-polarized phases. Near the 1.67 T quantum critical point, enhanced quantum fluctuations suggest conditions favorable for exotic excitations. These results establish KCeSe2 as a platform for exploring Kitaev physics on triangular lattices.

cond-mat.str-el↗

arXiv2025

GraphNarrator: Generating Textual Explanations for Graph Neural Networks

Graph representation learning has garnered significant attention due to its broad applications in various domains, such as recommendation systems and social network analysis. Despite advancements in graph learning methods, challenges still remain in explainability when graphs are associated with semantic features. In this paper, we present GraphNarrator, the first method designed to generate natural language explanations for Graph Neural Networks. GraphNarrator employs a generative language model that maps input-output pairs to explanations reflecting the model's decision-making process. To address the lack of ground truth explanations to train the model, we propose first generating pseudo-labels that capture the model's decisions from saliency-based explanations, then using Expert Iteration to iteratively train the pseudo-label generator based on training objectives on explanation quality. The high-quality pseudo-labels are finally utilized to train an end-to-end explanation generator model. Extensive experiments are conducted to demonstrate the effectiveness of GraphNarrator in producing faithful, concise, and human-preferred natural language explanations.

cs.LG↗

arXiv2025

CoderAgent: Simulating Student Behavior for Personalized Programming Learning with Large Language Models

Personalized programming tutoring, such as exercise recommendation, can enhance learners' efficiency, motivation, and outcomes, which is increasingly important in modern digital education. However, the lack of sufficient and high-quality programming data, combined with the mismatch between offline evaluation and real-world learning, hinders the practical deployment of such systems. To address this challenge, many approaches attempt to simulate learner practice data, yet they often overlook the fine-grained, iterative nature of programming learning, resulting in a lack of interpretability and granularity. To fill this gap, we propose a LLM-based agent, CoderAgent, to simulate students' programming processes in a fine-grained manner without relying on real data. Specifically, we equip each human learner with an intelligent agent, the core of which lies in capturing the cognitive states of the human programming practice process. Inspired by ACT-R, a cognitive architecture framework, we design the structure of CoderAgent to align with human cognitive architecture by focusing on the mastery of programming knowledge and the application of coding ability. Recognizing the inherent patterns in multi-layered cognitive reasoning, we introduce the Programming Tree of Thought (PTOT), which breaks down the process into four steps: why, how, where, and what. This approach enables a detailed analysis of iterative problem-solving strategies. Finally, experimental evaluations on real-world datasets demonstrate that CoderAgent provides interpretable insights into learning trajectories and achieves accurate simulations, paving the way for personalized programming education.

cs.AI↗

arXiv2025

Learning to Select In-Context Demonstration Preferred by Large Language Model

In-context learning (ICL) enables large language models (LLMs) to adapt to new tasks during inference using only a few demonstrations. However, ICL performance is highly dependent on the selection of these demonstrations. Recent work explores retrieval-based methods for selecting query-specific demonstrations, but these approaches often rely on surrogate objectives such as metric learning, failing to directly optimize ICL performance. Consequently, they struggle to identify truly beneficial demonstrations. Moreover, their discriminative retrieval paradigm is ineffective when the candidate pool lacks sufficient high-quality demonstrations. To address these challenges, we propose GenICL, a novel generative preference learning framework that leverages LLM feedback to directly optimize demonstration selection for ICL. Experiments on 19 datasets across 11 task categories demonstrate that GenICL achieves superior performance than existing methods in selecting the most effective demonstrations, leading to better ICL performance.

cs.LG↗

arXiv2025

DAPO: An Open-Source LLM Reinforcement Learning System at Scale

Inference scaling empowers LLMs with unprecedented reasoning ability, with reinforcement learning as the core technique to elicit complex reasoning. However, key technical details of state-of-the-art reasoning LLMs are concealed (such as in OpenAI o1 blog and DeepSeek R1 technical report), thus the community still struggles to reproduce their RL training results. We propose the $\textbf{D}$ecoupled Clip and $\textbf{D}$ynamic s$\textbf{A}$mpling $\textbf{P}$olicy $\textbf{O}$ptimization ($\textbf{DAPO}$) algorithm, and fully open-source a state-of-the-art large-scale RL system that achieves 50 points on AIME 2024 using Qwen2.5-32B base model. Unlike previous works that withhold training details, we introduce four key techniques of our algorithm that make large-scale LLM RL a success. In addition, we open-source our training code, which is built on the verl framework, along with a carefully curated and processed dataset. These components of our open-source system enhance reproducibility and support future research in large-scale LLM RL.

cs.LG↗

arXiv2025

MAGI-1: Autoregressive Video Generation at Scale

We present MAGI-1, a world model that generates videos by autoregressively predicting a sequence of video chunks, defined as fixed-length segments of consecutive frames. Trained to denoise per-chunk noise that increases monotonically over time, MAGI-1 enables causal temporal modeling and naturally supports streaming generation. It achieves strong performance on image-to-video (I2V) tasks conditioned on text instructions, providing high temporal consistency and scalability, which are made possible by several algorithmic innovations and a dedicated infrastructure stack. MAGI-1 facilitates controllable generation via chunk-wise prompting and supports real-time, memory-efficient deployment by maintaining constant peak inference cost, regardless of video length. The largest variant of MAGI-1 comprises 24 billion parameters and supports context lengths of up to 4 million tokens, demonstrating the scalability and robustness of our approach. The code and models are available at https://github.com/SandAI-org/MAGI-1 and https://github.com/SandAI-org/MagiAttention. The product can be accessed at https://sand.ai.

cs.CV↗

arXiv2025

Causal Effect of Functional Treatment

We study the causal effect with a functional treatment variable, where practical applications often arise in neuroscience, biomedical sciences, etc. Previous research concerning the effect of a functional variable on an outcome is typically restricted to exploring correlation rather than causality. The generalized propensity score, which is often used to calibrate the selection bias, is not directly applicable to a functional treatment variable due to a lack of definition of probability density function for functional data. We propose three estimators for the average dose-response functional based on the functional linear model, namely, the functional stabilized weight estimator, the outcome regression estimator and the doubly robust estimator, each of which has its own merits. We study their theoretical properties, which are corroborated through extensive numerical experiments. A real data application on electroencephalography data and disease severity demonstrates the practical value of our methods.

stat.ME↗

arXiv2025

Gapless spinon excitations emerging from a multipolar transverse field in the triangular-lattice Ising antiferromagnet NaTmSe2

The triangular-lattice quantum Ising antiferromagnet is a promising platform for realizing Anderson's quantum spin liquid, though finding suitable materials to realize it remains a challenge. Here, we present a comprehensive study of NaTmSe2 using magnetization, specific heat, neutron scattering, and muon spin relaxation, combined with theoretical calculations. We demonstrate that NaTmSe2 realizes the transverse field Ising model and quantitatively determine its exchange parameters. Our results reveal a multipolar spin-polarized state coexisting with a dipolar spin-disordered state. These states feature gapless spinon excitations mediated by the multipolar moments. The study shows how multiple types of magnetism can emerge in distinct magnetic channels (dipolar and multipolar) within a single magnet, advancing our understanding of spin-frustrated Ising physics and opening pathways for different quantum computing applications.

cond-mat.str-el↗

arXiv2025

Integrated Sensing and Communications for Pinching-Antenna Systems (PASS)

An integrated sensing and communication (ISAC) design for pinching antenna systems (PASS) is proposed, where the pinching antennas are deployed to establish reliable line-of-sight communication and sensing links. More particularly, a separated ISAC design is proposed for the two-waveguide PASS, where one waveguide is used to emit the information-bearing signals for ISAC transmission while the other waveguide is used to receive the reflected echo signals. Based on this framework, a penalty-based alternating optimization algorithm is proposed to maximize the illumination power as well as ensure the communication quality-of-service requirement. Numerical results demonstrate that the proposed PASS-ISAC scheme outperforms the conventional antenna scheme.

cs.IT↗

arXiv2025

Photonic Structures to Achieve High-Performance Dew-Harvesting in a 24-h Day-Night Cycle

Although most prior research on the dew-harvesting technology has focused on nocturnal operations, achieving round-the-clock freshwater harvesting remains crucial. However, daytime dew-harvesting faces two key challenges as compared to its nighttime counterpart: the high solar irradiance and the large contrast between the ambient temperature and the dewpoint. To address these challenges and guide the photonic design, we develop a theoretical framework to analyze dew-harvesting in a 24-h day-night cycle. Using Nanjing as an example, our analyses reveal that, in the solar regime, a minimum average solar reflectivity of 0.92 is required; in the infrared regime, a 10% reduction in absorptivity outside the 8-13 um transparency window is equivalent to a 5.9% enhancement in emissivity within the window. Guided by these findings, we propose a photonic design, which, in a synthetic experiment with measured meteorological datasets, achieves a water production rate of 313 g/m2-day, in which nearly 40% is contributed by daytime. This performance reaches approximately 70% of the theoretical maximum predicted using the ideal spectrum. We end by optimizing the layout of condensers in practical applications.

physics.optics↗

arXiv2025

Testing Conditional Independence via Density Ratio Regression

This paper develops a conditional independence (CI) test from a conditional density ratio (CDR) for weakly dependent data. The main contribution is presenting a closed-form expression for the estimated conditional density ratio function with good finite-sample performance. The key idea is exploiting the linear sieve combined with the quadratic norm. Matsushita et al. (2022) exploited the linear sieve to estimate the unconditional density ratio. We must exploit the linear sieve twice to estimate the conditional density ratio. First, we estimate an unconditional density ratio with an unweighted sieve least-squares regression, as done in Matsushita et al. (2022), and then the conditional density ratio with a weighted sieve least-squares regression, where the weights are the estimated unconditional density ratio. The proposed test has several advantages over existing alternatives. First, the test statistic is invariant to the monotone transformation of the data distribution and has a closed-form expression that enhances computational speed and efficiency. Second, the conditional density ratio satisfies the moment restrictions. The estimated ratio satisfies the empirical analog of those moment restrictions. As a result, the estimated density ratio is unlikely to have extreme values. Third, the proposed test can detect all deviations from conditional independence at rates arbitrarily close to $n^{-1/2}$ , and the local power loss is independent of the data dimension. A small-scale simulation study indicates that the proposed test outperforms the alternatives in various dependence structures.

stat.ME↗

arXiv2025

An Empirical Study on Prompt Compression for Large Language Models

Prompt engineering enables Large Language Models (LLMs) to perform a variety of tasks. However, lengthy prompts significantly increase computational complexity and economic costs. To address this issue, we study six prompt compression methods for LLMs, aiming to reduce prompt length while maintaining LLM response quality. In this paper, we present a comprehensive analysis covering aspects such as generation performance, model hallucinations, efficacy in multimodal tasks, word omission analysis, and more. We evaluate these methods across 13 datasets, including news, scientific articles, commonsense QA, math QA, long-context QA, and VQA datasets. Our experiments reveal that prompt compression has a greater impact on LLM performance in long contexts compared to short ones. In the Longbench evaluation, moderate compression even enhances LLM performance. Our code and data is available at https://github.com/3DAgentWorld/Toolkit-for-Prompt-Compression.

cs.CL↗

arXiv2025

NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens

Recent advancements in Large Language Models (LLMs) have pushed the boundaries of natural language processing, especially in long-context understanding. However, the evaluation of these models' long-context abilities remains a challenge due to the limitations of current benchmarks. To address this gap, we introduce NovelQA, a benchmark tailored for evaluating LLMs with complex, extended narratives. Constructed from English novels, NovelQA offers a unique blend of complexity, length, and narrative coherence, making it an ideal tool for assessing deep textual understanding in LLMs. This paper details the design and construction of NovelQA, focusing on its comprehensive manual annotation process and the variety of question types aimed at evaluating nuanced comprehension. Our evaluation of long-context LLMs on NovelQA reveals significant insights into their strengths and weaknesses. Notably, the models struggle with multi-hop reasoning, detail-oriented questions, and handling extremely long inputs, with average lengths exceeding 200,000 tokens. Results highlight the need for substantial advancements in LLMs to enhance their long-context comprehension and contribute effectively to computational literary analysis.

cs.CL↗

arXiv2025

The Other Side of the Coin: Exploring Fairness in Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by retrieving relevant document from external knowledge sources. By referencing this external knowledge, RAG effectively reduces the generation of factually incorrect content and addresses hallucination issues within LLMs. Recently, there has been growing attention to improving the performance and efficiency of RAG systems from various perspectives. While these advancements have yielded significant results, the application of RAG in domains with considerable societal implications raises a critical question about fairness: What impact does the introduction of the RAG paradigm have on the fairness of LLMs? To address this question, we conduct extensive experiments by varying the LLMs, retrievers, and retrieval sources. Our experimental analysis reveals that the scale of the LLMs plays a significant role in influencing fairness outcomes within the RAG framework. When the model scale is smaller than 8B, the integration of retrieval mechanisms often exacerbates unfairness in small-scale LLMs (e.g., LLaMA3.2-1B, Mistral-7B, and LLaMA3-8B). To mitigate the fairness issues introduced by RAG for small-scale LLMs, we propose two approaches, FairFT and FairFilter. Specifically, in FairFT, we align the retriever with the LLM in terms of fairness, enabling it to retrieve documents that facilitate fairer model outputs. In FairFilter, we propose a fairness filtering mechanism to filter out biased content after retrieval. Finally, we validate our proposed approaches on real-world datasets, demonstrating their effectiveness in improving fairness while maintaining performance.

cs.CL↗

arXiv2025

Enhancing Multi-task Learning Capability of Medical Generalist Foundation Model via Image-centric Multi-annotation Data

The emergence of medical generalist foundation models has revolutionized conventional task-specific model development paradigms, aiming to better handle multiple tasks through joint training on large-scale medical datasets. However, recent advances prioritize simple data scaling or architectural component enhancement, while neglecting to re-examine multi-task learning from a data-centric perspective. Critically, simply aggregating existing data resources leads to decentralized image-task alignment, which fails to cultivate comprehensive image understanding or align with clinical needs for multi-dimensional image interpretation. In this paper, we introduce the image-centric multi-annotation X-ray dataset (IMAX), the first attempt to enhance the multi-task learning capabilities of medical multi-modal large language models (MLLMs) from the data construction level. To be specific, IMAX is featured from the following attributes: 1) High-quality data curation. A comprehensive collection of more than 354K entries applicable to seven different medical tasks. 2) Image-centric dense annotation. Each X-ray image is associated with an average of 4.10 tasks and 7.46 training entries, ensuring multi-task representation richness per image. Compared to the general decentralized multi-annotation X-ray dataset (DMAX), IMAX consistently demonstrates significant multi-task average performance gains ranging from 3.20% to 21.05% across seven open-source state-of-the-art medical MLLMs. Moreover, we investigate differences in statistical patterns exhibited by IMAX and DMAX training processes, exploring potential correlations between optimization dynamics and multi-task performance. Finally, leveraging the core concept of IMAX data construction, we propose an optimized DMAX-based training strategy to alleviate the dilemma of obtaining high-quality IMAX data in practical scenarios.

cs.CV↗

arXiv2025

SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking

Recent advancements in large language models (LLMs) with billions of parameters have improved performance in various applications, but their inference processes demand significant energy and computational resources. In contrast, the human brain, with approximately 86 billion neurons, is much more energy-efficient than LLMs with similar parameters. Inspired by this, we redesign 7$\sim$70 billion parameter LLMs using bio-plausible spiking mechanisms, emulating the efficient behavior of the human brain. We propose the first spiking large language model, SpikeLLM. Coupled with the proposed model, two essential approaches are proposed to improve spike training efficiency: Generalized Integrate-and-Fire (GIF) neurons to compress spike length from $T$ to $\frac{T}{L} \log_2 L$ bits, and an Optimal Brain Spiking framework to divide outlier channels and allocate different $T$ for GIF neurons, which further compresses spike length to approximate $log_2T$ bits. The necessity of spike-driven LLM is proved by comparison with quantized LLMs with similar operations. In the OmniQuant pipeline, SpikeLLM reduces 11.01% WikiText2 perplexity and improves 2.55% accuracy of common scene reasoning on a LLAMA-7B W4A4 model. In the GPTQ pipeline, SpikeLLM achieves direct additive in linear layers, significantly exceeding PB-LLMs.

cs.LG↗

arXiv2025

LLM for Complex Reasoning Task: An Exploratory Study in Fermi Problems

Fermi Problems (FPs) are mathematical reasoning tasks that require human-like logic and numerical reasoning. Unlike other reasoning questions, FPs often involve real-world impracticalities or ambiguous concepts, making them challenging even for humans to solve. Despite advancements in AI, particularly with large language models (LLMs) in various reasoning tasks, FPs remain relatively under-explored. This work conducted an exploratory study to examine the capabilities and limitations of LLMs in solving FPs. We first evaluated the overall performance of three advanced LLMs using a publicly available FP dataset. We designed prompts according to the recently proposed TELeR taxonomy, including a zero-shot scenario. Results indicated that all three LLMs achieved a fp_score (range between 0 - 1) below 0.5, underscoring the inherent difficulty of these reasoning tasks. To further investigate, we categorized FPs into standard and specific questions, hypothesizing that LLMs would perform better on standard questions, which are characterized by clarity and conciseness, than on specific ones. Comparative experiments confirmed this hypothesis, demonstrating that LLMs performed better on standard FPs in terms of both accuracy and efficiency.

cs.CL↗

arXiv2025

Non-parametric Quantile Regression and Uniform Inference with Unknown Error Distribution

This paper studies the non-parametric estimation and uniform inference for the conditional quantile regression function (CQRF) with covariates exposed to measurement errors. We consider the case that the distribution of the measurement error is unknown and allowed to be either ordinary or super smooth. We estimate the density of the measurement error by the repeated measurements and propose the deconvolution kernel estimator for the CQRF. We derive the uniform Bahadur representation of the proposed estimator and construct the uniform confidence bands for the CQRF, uniformly in the sense for all covariates and a set of quantile indices, and establish the theoretical validity of the proposed inference. A data-driven approach for selecting the tuning parameter is also included. Monte Carlo simulations and a real data application demonstrate the usefulness of the proposed method.

stat.ME↗