arXiv subjects

Wei Cheng

Publications and source records attributed to Wei Cheng.

At least 109 records · Page 6Linked to original sources

arXiv2025

DISC: Dynamic Decomposition Improves LLM Inference Scaling

Inference scaling methods for LLMs often rely on decomposing problems into steps (or groups of tokens), followed by sampling and selecting the best next steps. However, these steps and their sizes are often predetermined or manually designed based on domain knowledge. We propose dynamic decomposition, a method that adaptively and automatically partitions solution and reasoning traces into manageable steps during inference. By more effectively allocating compute -- particularly through subdividing challenging steps and prioritizing their sampling -- dynamic decomposition significantly improves inference efficiency. Experiments on benchmarks such as APPS, MATH, and LiveCodeBench demonstrate that dynamic decomposition outperforms static approaches, including token-level, sentence-level, and single-step decompositions, reducing the pass@10 error rate by 5.0%, 6.7%, and 10.5% respectively. These findings highlight the potential of dynamic decomposition to improve a wide range of inference scaling techniques.

cs.LG↗

arXiv2025

$B_{(s)} \to S(a_0(1450), K_0^*(1430), f_0(1500))$ helicity form factors within QCD light-cone sum rules

In this paper, we investigate the helicity form factors (HFFs) of the $B_{(s)}$-meson decay into a scalar meson with a mass larger than 1~GeV, {\it i.e.,} $B \to a_0(1450)$, $B_{(s)} \to K_0^*(1430)$ and $B_{s} \to f_0(1500)$ by using light-cone sum rules approach. We take the standard currents for correlation functions. To enhance the precision of our calculations, we incorporate the next-to-leading order (NLO) corrections and retain the scalar meson twist-3 light-cone distribution amplitudes. Furthermore, we extend the HFFs to the entire physical $q^2$ region employing a simplified $z$-series expansion. At the point of $q^2=1\rm{~GeV^2}$, all NLO contributions to the HFFs are negative, with the maximum contribution around $25\%$. Then, as applications of these HFFs, we analyze the differential decay widths, branching ratios, and lepton polarization asymmetries for the semi-leptonic $B_{(s)} \to S \ell \barν_\ell$, FCNC $B_{(s)} \to S \ell \bar{\ell}$ and rare $B_{(s)} \to S ν\barν$ decays. Our results are consistent with existing studies within uncertainties. The current data still suffer from large uncertainties and need to be measured more precisely, which can lead to a better understanding of the fundamental properties of light scalar mesons.

hep-ph↗

arXiv2025

Global propagation of singularities for magnetic mechanical systems

We prove that singularities propagate globally for viscosity solutions of Hamilton-Jacobi equations related to magnetic mechanical systems on closed Riemannian manifolds. Our main result shows that for any weak KAM solution $u$, the singular set $\text{Sing}\,(u)$ remains invariant under the generalized gradient flow dynamics. The proof combines three key elements: (1) reduction from magnetic to Riemannian systems, (2) analysis of reparameterized flows, and (3) regularization techniques. Compared to previous analytic approaches, our geometric method provides clearer insights into the underlying Riemannian structure. We also establish necessary conditions for singularity existence, particularly when the Euler characteristic is nonzero and the magnetic form is non-exact. This approach does not extend directly to Finsler metrics due to structural differences.

math.AP↗

arXiv2025

Harnessing Vision Models for Time Series Analysis: A Survey

Time series analysis has witnessed the inspiring development from traditional autoregressive models, deep learning models, to recent Transformers and Large Language Models (LLMs). Efforts in leveraging vision models for time series analysis have also been made along the way but are less visible to the community due to the predominant research on sequence modeling in this domain. However, the discrepancy between continuous time series and the discrete token space of LLMs, and the challenges in explicitly modeling the correlations of variates in multivariate time series have shifted some research attentions to the equally successful Large Vision Models (LVMs) and Vision Language Models (VLMs). To fill the blank in the existing literature, this survey discusses the advantages of vision models over LLMs in time series analysis. It provides a comprehensive and in-depth overview of the existing methods, with dual views of detailed taxonomy that answer the key research questions including how to encode time series as images and how to model the imaged time series for various tasks. Additionally, we address the challenges in the pre- and post-processing steps involved in this framework and outline future directions to further advance time series analysis with vision models.

cs.LG↗

arXiv2025

Where's the liability in the Generative Era? Recovery-based Black-Box Detection of AI-Generated Content

The recent proliferation of photorealistic images created by generative models has sparked both excitement and concern, as these images are increasingly indistinguishable from real ones to the human eye. While offering new creative and commercial possibilities, the potential for misuse, such as in misinformation and fraud, highlights the need for effective detection methods. Current detection approaches often rely on access to model weights or require extensive collections of real image datasets, limiting their scalability and practical application in real world scenarios. In this work, we introduce a novel black box detection framework that requires only API access, sidestepping the need for model weights or large auxiliary datasets. Our approach leverages a corrupt and recover strategy: by masking part of an image and assessing the model ability to reconstruct it, we measure the likelihood that the image was generated by the model itself. For black-box models that do not support masked image inputs, we incorporate a cost efficient surrogate model trained to align with the target model distribution, enhancing detection capability. Our framework demonstrates strong performance, outperforming baseline methods by 4.31% in mean average precision across eight diffusion model variant datasets.

cs.LG↗

arXiv2025

Step1X-Edit: A Practical Framework for General Image Editing

In recent years, image editing models have witnessed remarkable and rapid development. The recent unveiling of cutting-edge multimodal models such as GPT-4o and Gemini2 Flash has introduced highly promising image editing capabilities. These models demonstrate an impressive aptitude for fulfilling a vast majority of user-driven editing requirements, marking a significant advancement in the field of image manipulation. However, there is still a large gap between the open-source algorithm with these closed-source models. Thus, in this paper, we aim to release a state-of-the-art image editing model, called Step1X-Edit, which can provide comparable performance against the closed-source models like GPT-4o and Gemini2 Flash. More specifically, we adopt the Multimodal LLM to process the reference image and the user's editing instruction. A latent embedding has been extracted and integrated with a diffusion image decoder to obtain the target image. To train the model, we build a data generation pipeline to produce a high-quality dataset. For evaluation, we develop the GEdit-Bench, a novel benchmark rooted in real-world user instructions. Experimental results on GEdit-Bench demonstrate that Step1X-Edit outperforms existing open-source baselines by a substantial margin and approaches the performance of leading proprietary models, thereby making significant contributions to the field of image editing.

cs.CV↗

arXiv2025

Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Search

Traditional reinforcement learning and planning typically requires vast amounts of data and training to develop effective policies. In contrast, large language models (LLMs) exhibit strong generalization and zero-shot capabilities, but struggle with tasks that require detailed planning and decision-making in complex action spaces. We introduce STRATEGIST, a novel approach that integrates the strengths of both methods. Our approach leverages LLMs to search and update high-level strategies (as text), which are then refined and executed by low-level Monte Carlo Tree Search (MCTS). STRATEGIST is a generalizable framework to optimize the strategy through population-based self-play simulations without the need for any training data. We demonstrate the effectiveness of STRATEGIST in learning optimal strategies for competitive, multi-turn games with partial information, including Game of Pure Strategy (GOPS) and multi-agent, hidden-identity discussion games like The Resistance: Avalon. Our results show that agents equipped with STRATEGIST outperform those trained with traditional RL methods, other LLM-based skill acquisition techniques, pre-existing LLM agents across both game environments and achieves comparable performance against human players.

cs.AI↗

arXiv2025

Observation of hierarchy of Hilbert space ergodicities in the quantum dynamics of a single spin

Ergodicity, the property that all allowed configurations are explored over time, plays a pivotal role in explaining the equilibrium behavior of classical dynamical systems. Yet, such a property is typically precluded in quantum systems owing to the presence of energy eigenstates, which are stationary states in dynamics. However, recent theoretical works have argued that ergodic explorations of the Hilbert space, occurring at varying levels as measured by statistical pseudorandomness of the time-evolved quantum states, may be exhibited for quantum systems driven by Hamiltonians with aperiodic time dependencies, which do not face such obstacles. Here, we experimentally investigate the hierarchy of Hilbert-space ergodicities (HSE) achievable in the dynamics of a single quantum spin realized by a solid-state defect in diamond, upon subjecting it to various time-dependent modulations. Through continuous monitoring of spin trajectories with full state tomography, different degrees of HSE were observed, ranging from no HSE in a time-periodic (Floquet) drive, to partial HSE in a smoothly kicked time-quasiperiodic drive, to complete HSE in a drive composed of a sequence of kicks generated by the Fibonacci word. We formulate a theoretical understanding of the increasing levels of HSE observed by attributing them to increasing levels of complexities associated with the drive sequences, whose notions we elucidate. Our work constitutes the first unambiguous experimental evidence of Hilbert space ergodicity and promotes deeper investigations into the mechanisms and fine-grained levels with which closed quantum systems reach equilibrium.

quant-ph↗

arXiv2025

OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation

Text-to-image (T2I) models have garnered significant attention for generating high-quality images aligned with text prompts. However, rapid T2I model advancements reveal limitations in early benchmarks, lacking comprehensive evaluations, for example, the evaluation on reasoning, text rendering and style. Notably, recent state-of-the-art models, with their rich knowledge modeling capabilities, show promising results on the image generation problems requiring strong reasoning ability, yet existing evaluation systems have not adequately addressed this frontier. To systematically address these gaps, we introduce OneIG-Bench, a meticulously designed comprehensive benchmark framework for fine-grained evaluation of T2I models across multiple dimensions, including prompt-image alignment, text rendering precision, reasoning-generated content, stylization, and diversity. By structuring the evaluation, this benchmark enables in-depth analysis of model performance, helping researchers and practitioners pinpoint strengths and bottlenecks in the full pipeline of image generation. Specifically, OneIG-Bench enables flexible evaluation by allowing users to focus on a particular evaluation subset. Instead of generating images for the entire set of prompts, users can generate images only for the prompts associated with the selected dimension and complete the corresponding evaluation accordingly. Our codebase and dataset are now publicly available to facilitate reproducible evaluation studies and cross-model comparisons within the T2I research community.

cs.CV↗

arXiv2025

Less is More: DocString Compression in Code Generation

The widespread use of Large Language Models (LLMs) in software engineering has intensified the need for improved model and resource efficiency. In particular, for neural code generation, LLMs are used to translate function/method signature and DocString to executable code. DocStrings which capture user re quirements for the code and used as the prompt for LLMs, often contains redundant information. Recent advancements in prompt compression have shown promising results in Natural Language Processing (NLP), but their applicability to code generation remains uncertain. Our empirical study show that the state-of-the-art prompt compression methods achieve only about 10% reduction, as further reductions would cause significant performance degradation. In our study, we propose a novel compression method, ShortenDoc, dedicated to DocString compression for code generation. Our extensive experiments on six code generation datasets, five open-source LLMs (1B to 10B parameters), and one closed-source LLM GPT-4o confirm that ShortenDoc achieves 25-40% compression while preserving the quality of generated code, outperforming other baseline methods at similar compression levels. The benefit of this research is to improve efficiency and reduce the cost while maintaining the quality of the generated code, especially when calling third-party APIs, and is able to reduce the token processing cost by 25-40%.

cs.SE↗

arXiv2025

Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers

While Diffusion Transformers (DiTs) have achieved breakthroughs in video generation, this long sequence generation task remains constrained by the quadratic complexity of attention mechanisms, resulting in significant inference latency. Through detailed analysis of attention maps in Video Diffusion Transformer (vDiT), we identify three recurring sparsity patterns: diagonal, multi-diagonal, and vertical-stripe structures. And even 3-6\% attention heads can be skipped. Crucially, these patterns exhibit strong layer-depth and head-position correlations but show limited dependence on the input content. Leveraging these findings, we propose Sparse-vDiT, a sparsity acceleration framework for vDiT comprising: 1) Pattern-optimized sparse kernels that replace dense attention with computationally efficient implementations for each identified sparsity pattern. 2) An offline sparse diffusion search algorithm that selects the optimal sparse computation strategy per layer and head via hardware-aware cost modeling. After determining the optimal configuration, we fuse heads within the same layer that share the same attention strategy, enhancing inference efficiency. Integrated into state-of-the-art vDiT models (CogVideoX1.5, HunyuanVideo, and Wan2.1), Sparse-vDiT achieves 2.09$\times$, 2.38$\times$, and 1.67$\times$ theoretical FLOP reduction, and actual inference speedups of 1.76$\times$, 1.85$\times$, and 1.58$\times$, respectively, while maintaining high visual fidelity, with PSNR values reaching 24.13, 27.09, and 22.59. Our work demonstrates that latent structural sparsity in vDiTs can be systematically exploited for long video synthesis.

cs.CV↗

arXiv2025

Footprint in fitting $B \to D$ vector form factor and determination for $D$-meson leading-twist LCDA

In this paper, we fit the $B\to D$ vector transition form factor (TFF) by using the data measured by \textsl{BABAR} and Belle Collaborations within Monte Carlo (MC) method. Meanwhile, the $B\to D$ TFF is also calculated by using the QCD light-cone sum rules approach (LCSRs) within right-handed chiral current correlation function. In the TFF, $D$-meson leading-twist light-cone distribution amplitude (LCDA) is the most important non-perturbative input parameter, the precise behavior of which is mainly determined by the parameter $B_{2;D}$ in the light-cone harmonic oscillator model. Through fitting the TFF, we determine the value $B_{2;D}=0.445$. Then, we present the curve of $D$-meson leading-twist LCDA in comparison with other theoretical approaches. Subsequently, the $B\to D$ TFF $f_{+}^{BD}(q^2)$ at the large recoil region is $f_{+}^{BD}(0)=0.648_{-0.063}^{+0.067}$, which is compared in detail with theoretical estimates and experimental measurements. Furthermore, we calculated the decay width and branching fractions of the Cabibbo-favored semileptonic decays $B\to D\ell^+ν_{\ell}$, which lead to the results $\mathcal{B}(B^0\to D^-\ell ^+ν_{\ell}) =(2.10_{-0.38}^{+0.44})\times 10^{-2}$ and $\mathcal{B}(B^+\to \bar{D}^0\ell ^+ν_{\ell}) =(2.26_{-0.41}^{+0.48})\times 10^{-2}$. Finally, we predict the CKM matrix element with two scenarios $|V_{cb}|_{\rm SR}=41.55_{-4.50}^{+4.91}\times 10^{-3}$ and $|V_{cb} |_{\rm MC}=41.47_{-2.66}^{+2.55 }\times 10^{-3}$ from $B^0\to D^-\ell^+ν_{\ell}$, $|V_{cb}|_{\rm SR}=40.54_{-3.80}^{+4.33}\times 10^{-3}$ and $|V_{cb} |_{\rm MC}=40.46_{-1.38}^{+1.40}\times 10^{-3}$ from $B^+\to \bar{D}^0\ell^+ν_{\ell}$ which are in good agreement with theoretical and experimental predictions.

hep-ph↗

arXiv2025

Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assets

While generative artificial intelligence has advanced significantly across text, image, audio, and video domains, 3D generation remains comparatively underdeveloped due to fundamental challenges such as data scarcity, algorithmic limitations, and ecosystem fragmentation. To this end, we present Step1X-3D, an open framework addressing these challenges through: (1) a rigorous data curation pipeline processing >5M assets to create a 2M high-quality dataset with standardized geometric and textural properties; (2) a two-stage 3D-native architecture combining a hybrid VAE-DiT geometry generator with an diffusion-based texture synthesis module; and (3) the full open-source release of models, training code, and adaptation modules. For geometry generation, the hybrid VAE-DiT component produces TSDF representations by employing perceiver-based latent encoding with sharp edge sampling for detail preservation. The diffusion-based texture synthesis module then ensures cross-view consistency through geometric conditioning and latent-space synchronization. Benchmark results demonstrate state-of-the-art performance that exceeds existing open-source methods, while also achieving competitive quality with proprietary solutions. Notably, the framework uniquely bridges the 2D and 3D generation paradigms by supporting direct transfer of 2D control techniques~(e.g., LoRA) to 3D synthesis. By simultaneously advancing data quality, algorithmic fidelity, and reproducibility, Step1X-3D aims to establish new standards for open research in controllable 3D asset generation.

cs.CV↗

arXiv2025

Less is More: Towards Green Code Large Language Models via Unified Structural Pruning

The extensive application of Large Language Models (LLMs) in generative coding tasks has raised concerns due to their high computational demands and energy consumption. Unlike previous structural pruning methods designed for classification models that deal with lowdimensional classification logits, generative Code LLMs produce high-dimensional token logit sequences, making traditional pruning objectives inherently limited. Moreover, existing single component pruning approaches further constrain the effectiveness when applied to generative Code LLMs. In response, we propose Flab-Pruner, an innovative unified structural pruning method that combines vocabulary, layer, and Feed-Forward Network (FFN) pruning. This approach effectively reduces model parameters while maintaining performance. Additionally, we introduce a customized code instruction data strategy for coding tasks to enhance the performance recovery efficiency of the pruned model. Through extensive evaluations on three state-of-the-art Code LLMs across multiple generative coding tasks, the results demonstrate that Flab-Pruner retains 97% of the original performance after pruning 22% of the parameters and achieves the same or even better performance after post-training. The pruned models exhibit significant improvements in storage, GPU usage, computational efficiency, and environmental impact, while maintaining well robustness. Our research provides a sustainable solution for green software engineering and promotes the efficient deployment of LLMs in real-world generative coding intelligence applications.

cs.SE↗

arXiv2025

Baichuan 2: Open Large-scale Language Models

Large language models (LLMs) have demonstrated remarkable performance on a variety of natural language tasks based on just a few examples of natural language instructions, reducing the need for extensive feature engineering. However, most powerful LLMs are closed-source or limited in their capability for languages other than English. In this technical report, we present Baichuan 2, a series of large-scale multilingual language models containing 7 billion and 13 billion parameters, trained from scratch, on 2.6 trillion tokens. Baichuan 2 matches or outperforms other open-source models of similar size on public benchmarks like MMLU, CMMLU, GSM8K, and HumanEval. Furthermore, Baichuan 2 excels in vertical domains such as medicine and law. We will release all pre-training model checkpoints to benefit the research community in better understanding the training dynamics of Baichuan 2.

cs.CL↗

arXiv2025

Leveraging Large Self-Supervised Time-Series Models for Transferable Diagnosis in Cross-Aircraft Type Bleed Air System

Bleed Air System (BAS) is critical for maintaining flight safety and operational efficiency, supporting functions such as cabin pressurization, air conditioning, and engine anti-icing. However, BAS malfunctions, including overpressure, low pressure, and overheating, pose significant risks such as cabin depressurization, equipment failure, or engine damage. Current diagnostic approaches face notable limitations when applied across different aircraft types, particularly for newer models that lack sufficient operational data. To address these challenges, this paper presents a self-supervised learning-based foundation model that enables the transfer of diagnostic knowledge from mature aircraft (e.g., A320, A330) to newer ones (e.g., C919). Leveraging self-supervised pretraining, the model learns universal feature representations from flight signals without requiring labeled data, making it effective in data-scarce scenarios. This model enhances both anomaly detection and baseline signal prediction, thereby improving system reliability. The paper introduces a cross-model dataset, a self-supervised learning framework for BAS diagnostics, and a novel Joint Baseline and Anomaly Detection Loss Function tailored to real-world flight data. These innovations facilitate efficient transfer of diagnostic knowledge across aircraft types, ensuring robust support for early operational stages of new models. Additionally, the paper explores the relationship between model capacity and transferability, providing a foundation for future research on large-scale flight signal models.

eess.SP↗

arXiv2025

FAVOR-Bench: A Comprehensive Benchmark for Fine-Grained Video Motion Understanding

Multimodal Large Language Models (MLLMs) have shown remarkable capabilities in video content understanding but still struggle with fine-grained motion comprehension. To comprehensively assess the motion understanding ability of existing MLLMs, we introduce FAVOR-Bench, comprising 1,776 videos with structured manual annotations of various motions. Our benchmark includes both close-ended and open-ended tasks. For close-ended evaluation, we carefully design 8,184 multiple-choice question-answer pairs spanning six distinct sub-tasks. For open-ended evaluation, we develop both a novel cost-efficient LLM-free and a GPT-assisted caption assessment method, where the former can enhance benchmarking interpretability and reproducibility. Comprehensive experiments with 21 state-of-the-art MLLMs reveal significant limitations in their ability to comprehend and describe detailed temporal dynamics in video motions. To alleviate this limitation, we further build FAVOR-Train, a dataset consisting of 17,152 videos with fine-grained motion annotations. The results of finetuning Qwen2.5-VL on FAVOR-Train yield consistent improvements on motion-related tasks of TVBench, MotionBench and our FAVOR-Bench. Comprehensive assessment results demonstrate that the proposed FAVOR-Bench and FAVOR-Train provide valuable tools to the community for developing more powerful video understanding models. Project page: \href{https://favor-bench.github.io/}{https://favor-bench.github.io/}.

cs.CV↗

arXiv2025

TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents

Time series data is essential in various applications, including climate modeling, healthcare monitoring, and financial analytics. Understanding the contextual information associated with real-world time series data is often essential for accurate and reliable event predictions. In this paper, we introduce TimeCAP, a time-series processing framework that creatively employs Large Language Models (LLMs) as contextualizers of time series data, extending their typical usage as predictors. TimeCAP incorporates two independent LLM agents: one generates a textual summary capturing the context of the time series, while the other uses this enriched summary to make more informed predictions. In addition, TimeCAP employs a multi-modal encoder that synergizes with the LLM agents, enhancing predictive performance through mutual augmentation of inputs with in-context examples. Experimental results on real-world datasets demonstrate that TimeCAP outperforms state-of-the-art methods for time series event prediction, including those utilizing LLMs as predictors, achieving an average improvement of 28.75% in F1 score.

cs.AI↗