arXiv subjects

Pei Zhang

Publications and source records attributed to Pei Zhang.

At least 19 recordsLinked to original sources

arXiv2026

CRESiST: Self-Enhancing Explorative Policy Learning for Simultaneous Speech Translation

Simultaneous Speech Translation (SimulST) requires balancing translation quality and latency. While Large Language Models (LLMs) excel in offline translation, current LLM-based SimulST systems predominantly rely on prefix-incremental generation. This paradigm suffers from redundant Key-Value cache recomputation and necessitates external Voice Activity Detection to process unbounded streams. To overcome these bottlenecks, we propose an end-to-end interleaved read-write framework built upon an omni-modal LLM. By interleaving speech chunks and text tokens, our architecture achieves full KV cache reuse, natively supporting segmentation-free, infinite-length inference. Furthermore, to learn optimal streaming policies without rigid offline alignments, we introduce a novel Actor-Critic self-enhancement pipeline. An RNN-T Actor dynamically samples read-write trajectories, while an LLM-based Critic evaluates these paths via intrinsic semantic and latency rewards, distilling the optimal policy back to the Actor. Extensive experiments on sentence-level (FLEURS, CoVoST2) and document-level (MuST-C, ACL 60/60) benchmarks demonstrate that our model establishes new state-of-the-art quality-latency Pareto frontiers, substantially outperforming existing cascaded and static-alignment baselines.

cs.CL↗

arXiv2026

Omnilingual-MsASR: Benchmarking Multimodal, Multilingual, Long-Form Multi-Speaker ASR of Omni Large Language Models

Omni-modal large language models (LLMs) have achieved remarkable progress in audio and video understanding. However, existing models struggle with long-form, multi-speaker videos, especially in understanding the entire video and recognizing who says what and when. To characterize the capability limit of current omni LLMs and guide future model development, we introduce Omnilingual-MsASR, a multilingual, multimodal benchmark for long-form Multi-speaker Automatic Speech Recognition (MsASR) covering six languages, recordings up to 30 minutes, and scenarios with up to 20 speakers. Omnilingual-MsASR provides parallel audio and video inputs to evaluate models' cross-modal consistency and their ability to exploit complementary cues across modalities. Extensive evaluations of leading omni-modal LLMs reveal substantial performance bottlenecks, while their tcpWERs frequently exceed 100%. These findings highlight fine-grained temporal localization, speaker disambiguation, and cross-modal synergy as key challenges for future omni-modal foundation models. Our work is available at https://github.com/QwenLM/Omnilingua-Bench/tree/main/Omnilingua-MSpeaker.

cs.CL↗

arXiv2026

Do Multi-Turn Conversational Capabilities Carry Over from Text to Speech? A Parallel Multilingual Benchmark

Successful multi-turn voice interaction requires models to retain information from earlier turns, follow persistent instructions, and track revisions to conversational state. These abilities are usually evaluated from transcripts, leaving open whether successes demonstrated from text carry over when the same turns are spoken, and whether this transfer is consistent across languages. We introduce Omnilingua-AudioMC, a strictly parallel multilingual extension of Audio MultiChallenge, to study this question. The benchmark contains 354 conversations and 1,404 rubric items across ten languages, with content, turn structure, cross-turn dependencies, and evaluation criteria aligned across transcript and speech conditions. Across six speech-language models, the shift from reference transcripts to direct speech input is strongly model- and language-dependent: models with similar transcript performance can diverge substantially under speech, and their relative strengths across languages can change. Paired rubric-level analysis shows that aggregate transcript-speech gaps can obscure which conversational requirements are preserved, because losses of transcript-demonstrated successes can be offset by gains on other items. Replacing preceding user turns, the current turn, or both with model-generated transcripts localizes recoverable degradation: for the two models with the largest gaps, history substitution yields a larger macro-level improvement than current-turn substitution in all ten languages, while the strongest direct-speech model gains little from either. These results show that transcript performance is an unreliable proxy for multilingual spoken conversational ability and identify the representation of preceding spoken context as an important source of recoverable degradation. Our code is publicly available at https://github.com/QwenLM/Omnilingua-Bench/tree/main/Omnilingua-AudioMC.

cs.CL↗

arXiv2026

A fast Laguerre--Fourier spectral method for the polyatomic Boltzmann collision operator with continuous internal energy

The polyatomic Boltzmann equation extends the classical kinetic description to gases whose molecules possess internal energy in addition to their translational motion. The distribution function then has one more variable than in the monatomic case, and the collision operator becomes an eight-fold integral whose evaluation dominates the cost of solving the equation. In this work, we propose a fast Laguerre--Fourier spectral method for the polyatomic Boltzmann equation with continuous internal energy, which evaluates the collision operator in $O(K^2MN^4\log N + K^3MN^4)$ online operations after a solution-independent precomputation, where $N$ is the number of Fourier modes in each velocity direction, $K$ is the number of Laguerre modes and $M$ is the number of quadrature points on the sphere. Specifically, the distribution function is expanded in generalized Laguerre functions in the internal energy and in Fourier modes in velocity. A sequence of changes of variables then separates the internal-energy integrals from the velocity convolutions, so that the former no longer involve the distribution function and are computed once and stored. Numerical experiments are presented for both the spatially homogeneous and the spatially inhomogeneous Boltzmann equation. They confirm the spectral accuracy of the proposed method and its agreement with the direct simulation Monte Carlo method.

math.NA↗

arXiv2026

A data-based conservative reduced collision model for the Boltzmann equation

The evaluation of the quadratic collision operator is one of the major computational bottlenecks in solving the Boltzmann equation owing to the high dimensionality and the nonlinear structure of the collision term. In this work, a conservative reduced collision (RC) model is developed within the discrete velocity framework. The bilinear collision operator is approximated through a kernel tensor preassembled with the problem-dependent reduced bases for the distribution function and the collision term. Consequently, online evaluation of the quadratic collision operator is reduced to tensor contractions. Exact discrete conservation of mass, momentum, and energy is enforced by constructing the collision basis in the orthogonal complement of the space spanned by the discrete collision invariants. Several numerical experiments validate the efficiency of this RC model.

math.NA↗

arXiv2026

Covertness as a resource constraint in quantum target sensing

Quantum covert target sensing aims to detect low-reflectivity targets embedded in the thermal environment while avoiding discovery by an adversary. This covertness requirement constrains the admissible probe energy, thereby imposing a trade-off between covertness and sensing performance. However, how this resource constraint determines the ultimate estimation precision of the target parameter remains largely unexplored. To fill this gap, we establish a unified framework connecting covert target discrimination with quantum-enhanced parameter estimation. We first present perturbative $ε$-covertness-induced bounds on the probe energy, which yield an admissible energy window narrowing as $\mathcal{O}(\sqrt{ε/M})$. Moreover, we introduce the covertness-constrained quantum Fisher information (CCQFI) to quantify the ultimate precision of target-reflectivity estimation and assess its attainability with specific measurements. For Gaussian-distributed coherent-state (GCS) and two-mode squeezed vacuum (TMSV) probes, their CCQFIs scale as $\mathcal{O}(ε)$ and $\mathcal{O}(M)+\mathcal{O}(\sqrt{εM})+\mathcal{O}(ε)$, respectively. In particular, in the strong covertness regime $ε\to0$, the GCS CCQFI vanishes, whereas the TMSV retains a finite $\mathcal{O}(M)$ contribution exhibiting a pronounced metrological advantage. Besides, photon counting saturates the GCS CCQFI, whereas homodyne detection achieves a higher attainability for the TMSV probe. Our work identifies covertness as a quantitative resource constraint governing the metrological limits of quantum target sensing.

quant-ph↗

arXiv2026

MuLA-Bench: A Multilingual Long-Form Audio Understanding Benchmark via Multi-Tier Auditing

Long-form audio performance is often summarized by context length and aggregate accuracy, obscuring how language, evidence, and task jointly shape difficulty. We introduce MuLA-Bench: 5,038 open-ended questions over 1,769 in-the-wild recordings totaling 1,377.9 hours, covering 16 languages and eight domains. A balanced Language x Domain semantic track supports controlled comparisons, while a complementary acoustic track preserves naturally occurring non-speech evidence. Evidence-grounded generation, shortcut checks, and language-expert review provide auditable questions without translating a shared source set or injecting target sounds. We evaluate ten audio-language models and conduct pooled diagnostics on a fixed eight-model cohort. Language rankings change across domains and tasks; acoustic-semantic performance gaps vary with the requested operation; and temporal errors can persist after the correct event is identified. Long-range retrieval is comparatively strong, while precise clock alignment and factual grounding of natural acoustic events remain fragile. MuLA-Bench thus exposes conditional failure patterns that a single long-context score does not capture.

cs.SD↗

arXiv2026

Comparing Tobit and Two-Part Hurdle Models for Semi-Continuous Longitudinal Data with an Application to Clonal Hematopoiesis

Zero-inflated nonnegative continuous longitudinal data frequently arise in biomedical studies where outcomes consist of a mixture of excess zeros and positive continuous measurements. Two widely used approaches for analyzing such data are mixed-model versions of Tobit and the two-part hurdle models. The Tobit assumes a latent regression model that is censored below a specified threshold, while the hurdle separately models the continuous positive outcomes and the binary indicator of being positive. The choice between these models has rarely been systematically discussed, and inappropriate model choice may lead to biased estimation and misleading scientific interpretations. In this paper, we derive rigorous mathematical conditions under which the two models are equivalent and show that the Tobit can be viewed as a special case of the hurdle model when the link function for the binary process is probit. Based on simulation studies, we found that the hurdle is more flexible and robust than the Tobit model. On the other hand, if the assumptions of the Tobit are met, this model is easier to interpret since it does not require distinct inferences on both the continuous and binary processes. We therefore recommend that the Tobit model only be used when these assumptions are scientifically plausible and empirically supported; otherwise, the hurdle model is preferable. We applied both models to study the dynamics of somatic mosaicism using longitudinal clonal fraction measurements from the Prostate, Lung, Colorectal, and Ovarian study data while accounting for excess zero values. Estimates obtained from the Tobit and hurdle models were broadly consistent with those from a standard linear model that ignored zero inflation. These findings provide additional support for previously reported associations in studies of clonal hematopoiesis across different types of mosaic chromosomal alterations.

stat.AP↗

arXiv2026

Accelerated iterative method for solving the steady-state Boltzmann equation

The efficient simulation of steady-state rarefied gas flows remains a significant computational challenge due to the high dimensionality of the collision integral and the severe numerical stiffness in the near-continuum regime. In this work, we propose a modified Newton method equipped with a macroscopic synthetic system (Newton-MS) for the steady-state Boltzmann equation with the quadratic collision operator. In Newton-MS, the modified Newton iteration is utilized as the outer nonlinear solver, while each Newton correction equation is solved by an inner source iteration, where the linearized collision operator is utilized to approximate the quadratic collision model, and it is reduced into a linear iteration. Moreover, a macroscopic synthetic system based on Chapman-Enskog closure is derived to accelerate the convergence of the linear inner iteration in the continuum limit. Besides, the fully discrete macroscopic synthetic system is deduced under the framework of the discontinuous Galerkin method to reduce computational cost compared to directly discretizing the continuous macroscopic synthetic system. Several numerical examples, including the 1D Fourier, Couette flow problem, and the 2D cavity flow and thermal-driven cavity flow, are studied to validate the high efficiency of Newton-MS.

math.NA↗

arXiv2026

See Silhouettes in Motion with Neuromorphic Vision

Quasi-bimodal objects, such as text, road signs, and barcodes, play a basic yet vital role in daily visual communication. By boiling these down to clear silhouettes, binarization uses a minimal language to convey essential vision cues for maximum downstream efficiency, especially for tasks that require simple geometric, topological reasoning rather than heavy appearance modeling. The catch is that frame-based imaging often struggles on mobile platforms like drones, self-driving cars, and underwater vehicles, in which rapid motion causes severe motion blur and harsh lighting washes out scene details. To overcome these physical limits, neuromorphic vision via event cameras, featuring microsecond time resolution and high dynamic range, steps in as a natural solution. Building upon this event-driven paradigm, we propose a simple yet effective dual-modal approach that harnesses the synergy between frames and events for training-free, real-time, high-frame-rate binarization on CPU-only devices. Extensive evaluations show that it earns competitive performance against leading techniques in reducing blur artifacts and delivers impressive improvements under challenging illumination at a lower computational cost. Besides, its asynchronous nature bypasses long-standing event-scarcity issues that break traditional time-binning reconstruction at fixed time slots, maintaining clear target shapes even at extreme kilohertz frame rates. Its binary results further serve as reliable representations to facilitate a range of downstream tasks. This work paves the way towards lightweight perception and interaction in embodied intelligence on resource-constrained edge platforms.

eess.IV↗

arXiv2026

On-Device Robotic Planning: Eliminating Inference Redundancy for Efficient Decision-Making

Reasoning-based robotic policies using large language and vision-language models achieve strong semantic planning capabilities but mostly suffer from a high inference latency that limits practical real-time deployment. In this work, we observe that robotic reasoning workloads contain substantial temporal redundancy, where consecutive observations frequently produce identical actions and subgoals. Based on this insight, we present REIS, a human cognition inspired robotic decision-making framework that minimizes unnecessary reasoning while preserving semantic adaptability. REIS combines lightweight scene gating, KV-steered affordance routing, and deliberative reasoning to accelerate robotic control under embodied constraints. Experiments on ALFRED, and real-world robotic tasks demonstrate that REIS significantly suppresses reasoning overhead while maintaining competitive task performance.

cs.RO↗

arXiv2026

LISA: Language-guided Interference-aware Spatial-Frequency Attention for Driver Gaze Estimation

Driver gaze estimation serves as a fundamental metric for evaluating driver attentiveness in modern monitoring systems. Beyond being vulnerable to sudden lighting changes and sensor noise, spatial-domain models struggle to disentangle authentic gaze cues from irrelevant visual attributes. In this paper, we propose LISA, a \textbf{L}anguage-guided \textbf{I}nterference-aware \textbf{S}patial-Frequency \textbf{A}ttention framework that combines frequency-domain priors with vision-language knowledge. Observing that the amplitude spectrum remains relatively stable even under spatial perturbations, we design a dual-domain fusion mechanism. It integrates stable low-frequency semantics into high-frequency details, employing spatial attention to precisely target ocular regions. To reduce semantic ambiguity, we also introduce a training-time disentanglement strategy. Using a frozen CLIP encoder and orthogonal regularization, we explicitly separate gaze features from appearance interference. Experiments on two benchmarks show that LISA achieves state-of-the-art performance, with significantly improved robustness against occlusions and lighting variations. The code repository is available at https://github.com/Mason-bupt/LISA.

cs.CV↗

arXiv2026

Aquatic Neuromorphic Optical Flow

Underwater environments impose severe constraints on conventional imaging systems and demand solutions that balance high-quality sensing with strict resource efficiency. While emerging event cameras offer a promising alternative, their potential in aquatic scenarios remains largely unexplored. Through the lens of neuromorphic vision, this work pioneers the investigation of motion fields that serve as key media for agile underwater perception. Built upon spiking neural networks, we introduce a self-supervised framework to estimate per-pixel optical flow from asynchronous event streams, elegantly bypassing the long-standing bottleneck of underwater data scarcity. Extensive evaluations demonstrate that our method achieves competitive visual and quantitative results against leading techniques while operating with superior computational efficiency. By bridging neuromorphic sensing and aquatic intelligence, this work opens new frontiers for lightweight, real-time, and low-cost perception on resource-constrained underwater edge platforms.

cs.CV↗

arXiv2026

Floquet quantum multiparameter estimation with periodic-driving-induced topological phase transition

Periodically driven systems provide a powerful platform for quantum multiparameter estimation. Constructing a static effective Hamiltonian in a proper rotating frame is commonly employed to assess the attainable precision. However, such an approach becomes nonfeasible for more general time-periodically driven systems. To tackle this dilemma, we develop a quantum multiparameter estimation strategy in the Floquet theory framework. The contributions of Floquet eigenmodes, quasienergies, and multi-photon processes to the quantum Fisher information matrix and measurement incompatibility are determined, respectively. Moreover, this approach is applied to a ring-shaped Rashba spin-orbit interferometer model exhibiting the topological phase transition (TPT). In the vicinity of the TPT boundary, we reveal a pronounced enhancement in the estimation precision of multiple parameters with the Heisenberg limit scaling and even higher. Meanwhile, the measurement incompatibility vanishes in an oscillatory manner, and the stroboscopic projective measurement enables the highest estimation precision achievable. This work provides a complete Floquet picture for time-dependent critical quantum multiparameter estimation.

quant-ph↗

arXiv2026

Direct Simultaneous Translation Activation for Large Audio-Language Models

Simultaneous speech-to-text translation (Simul-S2TT) aims to translate speech into target text in real time, outputting translations while receiving source speech input, rather than waiting for the entire utterance to be spoken. Simul-S2TT research often modifies model architectures to implement read-write strategies. However, with the rise of large audio-language models (LALMs), a key challenge is how to directly activate Simul-S2TT capabilities in base models without additional architectural changes. In this paper, we introduce {\bf Simul}taneous {\bf S}elf-{\bf A}ugmentation ({\bf SimulSA}), a strategy that utilizes LALMs' inherent capabilities to obtain simultaneous data by randomly truncating speech and constructing partially aligned translation. By incorporating them into offline SFT data, SimulSA effectively bridges the distribution gap between offline translation during pretraining and simultaneous translation during inference. Experimental results demonstrate that augmenting only about {\bf 1\%} of the simultaneous data, compared to the full offline SFT data, can significantly activate LALMs' Simul-S2TT capabilities without modifications to model architecture or decoding strategy.

cs.SD↗

arXiv2026

Electric Vehicle User Charging Behavior Analysis Integrating Psychological and Environmental Factors: A Statistical-Driven LLM based Agent Approach

With the growing adoption of electric vehicles (EVs), understanding user charging behavior has become critical for grid stability and transportation planning. This study investigates the behavioral heterogeneity of EV taxi drivers by analyzing the interaction between psychological traits and situational triggers within dynamic travel contexts. Leveraging large language models (LLMs) as a core simulation tool, a novel framework with statistical enhancement is developed to replicate and analyze the charging behaviors of taxi drivers. LLMs simulate personalized decision-making processes by leveraging natural language reasoning and role-playing capabilities, accounting for factors such as time sensitivity, price awareness, and range anxiety. Simulation results indicate that the framework reliably reproduces real-world charging behaviors across multiple urban environments. his fidelity arises from integrating statistical priors into the reasoning process, allowing the model to anchor its decisions in empirical behavioral patterns. Further analysis highlights the joint influence of environmental and psychological variables on charging decisions and reveals the heterogeneity of different user groups. The findings provide new insights into EV user behavior, offering a foundation for optimizing charging infrastructure, informing energy policy, and advancing the integration of EV behavioral models into smart transportation and energy management systems.

cs.AI↗

arXiv2026

Emergent aperiodicity in Bose-Bose mixtures induced by spin-dependent periodic potentials

We study the ground-state and low-lying metastable phases of repulsive binary Bose-Einstein condensates confined in twisted, spin-dependent periodic optical lattices. For balanced mixtures, weak intercomponent interactions yield a fourfold momentum-space symmetry dictated by the lattice geometry. Increasing the coupling strength leads to the emergence of additional momentum peaks that combine with the lattice-induced structure to produce an eightfold rotationally symmetric pattern, signaling quasicrystalline order. At intermediate interactions, global phase separation suppresses this quasicrystalline state; however, at stronger coupling, local phase separation gives rise to a long-lived metastable phase in which the eightfold symmetry is restored. In this regime, a secondary ring of dominant momentum peaks appears at smaller wave vectors, indicating longer-wavelength density modulations and a crossover from lattice-dominated to interaction-driven quasicrystalline order. In contrast, imbalanced mixtures form partially miscible density clusters with eightfold-symmetric aperiodic patterns only at intermediate coupling, while stronger interactions drive global phase separation and permanently destroy quasicrystalline order. Real-time simulations demonstrate that these aperiodic structures are dynamically stable and experimentally accessible. Our results show that quasicrystalline order can emerge in binary condensates without explicitly aperiodic lattices and reveal population balance as a key ingredient for stabilizing quantum quasicrystals.

cond-mat.quant-gas↗

arXiv2026

Qwen3-ASR Technical Report

In this report, we introduce Qwen3-ASR family, which includes two powerful all-in-one speech recognition models and a novel non-autoregressive speech forced alignment model. Qwen3-ASR-1.7B and Qwen3-ASR-0.6B are ASR models that support language identification and ASR for 52 languages and dialects. Both of them leverage large-scale speech training data and the strong audio understanding ability of their foundation model Qwen3-Omni. We conduct comprehensive internal evaluation besides the open-sourced benchmarks as ASR models might differ little on open-sourced benchmark scores but exhibit significant quality differences in real-world scenarios. The experiments reveal that the 1.7B version achieves SOTA performance among open-sourced ASR models and is competitive with the strongest proprietary APIs while the 0.6B version offers the best accuracy-efficiency trade-off. Qwen3-ASR-0.6B can achieve an average TTFT as low as 92ms and transcribe 2000 seconds speech in 1 second at a concurrency of 128. Qwen3-ForcedAligner-0.6B is an LLM based NAR timestamp predictor that is able to align text-speech pairs in 11 languages. Timestamp accuracy experiments show that the proposed model outperforms the three strongest force alignment models and takes more advantages in efficiency and versatility. To further accelerate the community research of ASR and audio understanding, we release these models under the Apache 2.0 license.

cs.CL↗