arXiv subjects

Chao Zhang

Publications and source records attributed to Chao Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2026

Time-JEPA: Grounding Video World Models with Physical Time Series

Video world models learn how visual environments evolve by predicting missing or future content. However, physical-state information can provide useful context for predicting how environments evolve. Prior work incorporates this information through different mechanisms: domain-specific physical laws can complicate multi-domain training, while supervision on individual frames or frame pairs gives limited attention to how physical state evolves over time. We introduce Time-JEPA, which fine-tunes a video world model using low-dimensional, temporally aligned physical time series as auxiliary supervision. Physical time series connect video representations with physical state at corresponding temporal positions, encouraging the model to preserve how physical state evolves. In multiple domains including autonomous driving, robot manipulation, and human motion, Time-JEPA improves current-state decoding, multi-horizon future-state prediction, and kinematic reconstruction over matched video-fine-tuned controls across both condition and state-variable distribution shifts. Multi-domain training provides aggregate gains over matched single-domain training in all three tasks. External evaluations further suggest that time-series supervision improves sensitivity to physical anomalies and the representation of physical interactions, including in unseen scenarios. Aggregate gains on a second video backbone indicate that the method can be applied to more backbones. These results suggest that physical time series help video world models better understand the physical world.

cs.CV↗

arXiv2026

Adaptive Patching Is Harder Than It Looks For Time-Series Forecasting

Adaptive patching has been proposed for time-series forecasting: allocate finer patches where the sequence looks locally informative. We compare three adaptive patching methods with tuned uniform counterparts that use one fixed patch size throughout the input, while preserving each method's backbone family and shared training protocol. Across the evaluated long-horizon benchmarks, we observe mostly small accuracy differences and no consistent advantage for adaptive patching; uniform variants generally train faster and avoid adaptive routing at inference. Motivated by these observations, we develop a theoretical framework for budgeted patch allocation through a common resolution-density representation of uniform, nonuniform, and overlapping layouts, independently of the patching rule. Within a separable shared-loss model, we derive an exact criterion for when nonuniform allocation improves on uniform allocation at the same budget. We characterize modeled gains locally through a quadratic surrogate and bound them globally under strong convexity and a bounded loss derivative. The analysis distinguishes where resolution is placed from how much it is redistributed: even an allocation that correctly ranks regions by their weights can redistribute too much resolution and increase modeled loss. Without coupling across time, positive regional weights preserve the minimizers of the shared loss. When the shared loss has a common minimizing rate, the remaining absolute headroom for improvement tends to zero as the uniform rate approaches it. These results characterize the roles of regional weighting, placement, and redistribution strength under the stated model. The empirical findings support evaluating adaptive patching against a tuned uniform baseline and weighing any accuracy gain against the additional training and inference cost of adaptive routing.

cs.LG↗

arXiv2026

Universal entanglement detection with geometrically optimal witnesses

Constructing optimal entanglement witnesses for arbitrary quantum systems remains a formidable challenge, hindered by the structural complexity of high-dimensional quantum state spaces. In this work, we overcome this barrier by introducing a unified framework for constructing geometrically optimal witnesses capable of identifying arbitrary entanglement structures. By leveraging the geometric properties of a target quantum state relative to a specific entanglement structure in Hilbert-Schmidt space, these witnesses are constructed in an efficient and effective manner. Among all nontrivial witnesses, they provide maximal distinguishability between the target state and the set of states without the desired entanglement structure. As a result, our approach enables the universal detection of bipartite, multipartite, and high-dimensional entanglement. To demonstrate its experimental feasibility, we apply the framework to a four-qubit nuclear spin quantum system. Compared to existing methods, our approach offers enhanced noise robustness, broader detection universality, and improved construction efficiency, providing a powerful and versatile tool for advancing the characterization and understanding of entanglement in quantum systems.

quant-ph↗

arXiv2026

ProMoDD: Dataset Distillation for 3D Human Motion via Prototype-Conditioned Diffusion Models

Dataset distillation (DD) aims to retain the training utility of a large dataset in a much smaller set of examples. For skeleton-based action recognition, this requires capturing discriminative motion variation without disrupting the geometry and temporal continuity of articulated bodies. Directly optimizing joint coordinates can violate these constraints, while plausible motion generation alone does not ensure useful training examples. We propose Prototype-Conditioned Motion Dataset Distillation (ProMoDD), which conditions a motion diffusion model on class-wise prototypes of temporally resolved recognition features. We further tailor diffusion training to motion data, enabling the generator to model both skeletal configurations and their temporal evolution. We then construct each compact training set through budget-aware hybrid selection: representative real sequences cover densely populated regions of the pooled recognition space, while selected generated candidates complement them in less-populated regions. This couples prototype-guided generation with feature-space selection, balancing the reliability of observed motions with the variation of synthesis under a fixed per-class sequence budget. Across five difficulty-stratified subsets of the single-person classes in NTU RGB+D 60, ProMoDD achieves its largest gains under tight distillation budgets, reaching 40.32% top-1 accuracy on Full49 with three sequences per class, 4.57 percentage points above class-wise k-means. These gains reflect the complete framework. The same distilled sets train ST-GCN, MS-G3D, and CTR-GCN without regeneration.

cs.CV↗

arXiv2026

Analytic Electromagnetic Effective Source for Geodesic Orbit in Schwarzschild Spacetime

Self-force effects are essential for accurately describing the motion of small bodies in curved spacetime. The effective-source approach privides a promising route to higher-order self-force calculations. In this paper, we construct analytic electromagnetic effective source for geodesic motion in Schwarzschild spacetime. An angular completion and a tetrad decomposition express the vector-harmonic puncture modes as finite sums of scalar modes. We derive the coupling coefficients for both parity sectors and obtain the effective sources from the Maxwell mode equations. The point-particle delta contribution cancels, and representative eccentric-orbit results show finite mode sources near the particle. These expressions provide the source terms for future $1+1$-dimensional residual-field calculations.

gr-qc↗

arXiv2026

Bayesian Speech Synthesizers Can Learn from Multiple Teachers

Text-to-speech (TTS) is a "one-to-many" problem: one text can be spoken in many valid ways, so its realizations form a distribution whose spread depends on the input. Most TTS models do not model this spread explicitly: they leave it implicit in the sampling process or fix it in advance. To model this distribution explicitly, we propose BELLE (Bayesian evidential learning with language modelling), which places a Bayesian prior over the mean and variance of each mel-spectrogram frame and predicts this prior from the input, while keeping the architecture and single-pass sampling of common continuous-valued autoregressive TTS models such as MELLE. Estimating this variance requires several recordings of each text, whereas standard TTS corpora provide one. We therefore construct two parallel-speech corpora: ParallelVox from real human recordings and LS+6TTS, which adds speech from six pre-trained TTS teachers to LibriSpeech (LS). Trained on these corpora, BELLE outperforms MELLE under the same training conditions, is competitive with larger TTS systems on the objective metrics of LibriSpeech test-clean, and supports streaming generation. Analyses of repeated human readings show that real speech has the properties the Bayesian prior assumes and that the variance predicted by BELLE follows the variation across these readings. Parallel speech improves BELLE, and synthetic parallel speech performs comparably to real parallel speech at the same volume and is far easier to scale.

cs.SD↗

arXiv2026

OWLET: On-Policy Weighted Learning via Evidence Transfer for Video Large Language Model Post-Training

Post-training has become a key stage for eliciting reasoning capabilities in Video Large Language Models (Video-LLMs). Existing post-training methods mainly follow two paradigms: reinforcement learning (RL) and on-policy distillation (OPD). However, they do not fully exploit visual evidence for effective training. RL supervises the policy with a single trajectory-level reward, which is too coarse-grained for video reasoning: the same response-level advantage is applied to every generated token, disregarding that the tokens of a video response are grounded in the visual stream to very different degrees. OPD provides token-level supervision through output-distribution matching, but lacks direct supervision of the spatiotemporal representations that produce those outputs. We observe across extensive experiments that the same model reasons more accurately when its video view combines global temporal context with question-relevant details. Comparing the same student-generated response under this view and uniform sampling reveals changes in token likelihoods and internal representations, providing a training signal for evidence use. Building on this observation, we propose OWLET, an evidence-privileged RL method that derives training signals from a richer teacher view. This view couples uniform global context with question-conditioned local windows, providing visual evidence without requiring a larger model. To better exploit this privileged visual evidence, OWLET introduces (i) \textbf{evidence-aware credit redistribution}, which compares token likelihoods across the two views and redistributes positive response-level advantages according to their relative evidence support; and (ii) \textbf{spatiotemporal state alignment}, which aligns the two views' hidden states on verifier-accepted responses.

cs.CV↗

arXiv2026

The Silk Nebula: A Newly Discovered Giant [OIII] Emission Filament

We report [O\,\textsc{iii}] narrow-band imaging of an extremely large-scale nebula, the Silk Nebula, discovered by the SWONS (Super Wide-Field [O\,\textsc{iii}] Narrow-band Survey), together with multi-fiber medium-resolution spectroscopic data covering the H$α$, [N\,\textsc{ii}], [S\,\textsc{ii}], and [O\,\textsc{iii}] emission lines, acquired with the Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST) as part of the Medium-Resolution Spectroscopic Survey of Nebulae (MRS-N). Based on 2,684 high-quality spectra, we measured its physical parameters, including relative line intensities, radial velocities (RVs), and full widths at half-maximum (FWHMs). The [S\,\textsc{ii}]/H$α$ ratio of the nebula significantly exceeds the shock-excitation threshold of 0.4. Its location in BPT diagnostic diagrams and spatial morphology of [O\,\textsc{iii}] both support the optical emission is dominated by shock-heated gas. The oxygen abundance is near solar and spatially uniform, consistent with a shock propagating into the ambient interstellar medium. Because [O\,\textsc{iii}] is the only emission line that clearly traces the nebular morphology, we base our distance estimate exclusively on the [O\,\textsc{iii}] kinematics, obtaining a fiducial distance of $\sim$160 pc. At this distance, the nebula has a physical extent of $\sim$28 pc along the north-south direction. We suggest that the Silk Nebula is a large [O\,\textsc{iii}] filament possibly associated with a supernova remnant, although a diffuse-ionized-gas (DIG) or ionized-bubble origin cannot be ruled out with the present data, and one of the largest angular-scale [O\,\textsc{iii}] emission structures documented to date. Future deep X-ray observations and shock-modeling efforts are required to firmly establish its nature.

astro-ph.GA↗

arXiv2026

Randomized Scores and Diverse Timbres: Augmenting Automatic Music Transcription with Online-Generated Data

Automatic music transcription (AMT) is limited by the scarcity of audio recordings paired with precise symbolic annotations. Synthetic data can provide supervision at scale, but it remains unclear whether effective transfer depends on realistic score structure or broad timbral coverage. We study these factors separately through an online sampler--renderer pipeline. A unified corruption sampler ranges from unmodified MIDI clips through partial corruption to deeply randomized note-event distributions. The renderer converts these events to audio while independently controlling instrument and timbral coverage. A fixed transcription model is trained jointly on offline recordings and newly rendered examples. Controlled ablations reveal an asymmetry between the two factors: moderate corruption of the note-event distribution does not impair transfer and can improve it, whereas broader renderer-side timbral support consistently improves out-of-domain generalization under a fixed note-event distribution. Finally, online-rendered examples complement real and existing synthetic data in a strong combined-data regime. These results suggest that synthetic AMT data should prioritize coverage of note-level attributes and their timbral realizations over realistic joint score structure.

eess.AS↗

arXiv2026

PERSIST: Who-What-When Memory Across Sessions for Full-Duplex Spoken Dialogue

Modern voice assistants may be shared by multiple users and should be able to answer questions about earlier conversations such as "When did I originally plan to leave?" or adapt their behavior to individual users based on past interactions. This requires more than retrieving a topically similar passage: the assistant must identify the current speaker, recover the relevant past state, and distinguish it from later revisions. We present PERSIST, a persistent memory system for multi-session, multi-speaker spoken dialogue that explicitly models Who, What, and When. PERSIST structures cross-session histories into readable event records and retrieves them with a 3W joint scoring mechanism that combines semantic content, acoustic speaker identity, and temporal state. For real-time full-duplex interaction, PERSIST further reuses intermediate representations from the dialogue backbone, avoiding query-audio re-encoding and reducing retrieval latency from 578.42 ms to 7.03 ms. We also introduce SpokenTrace, a diagnostic benchmark that factorizes evaluation along memory tasks and speaker-query types, exposing failures in recall, speaker attribution, and temporal-state tracking. On SpokenTrace, PERSIST achieves 85.08% end-to-end task accuracy and improves all-support EM@3 from 49.01% with BGE-large to 82.10%.

cs.AI↗

arXiv2026

Multiplexing Neural Audio Watermarks with Adaptive Routing

Audio watermarking supports speech authenticity verification. We study whether combining heterogeneous watermarks can retain at least one detectable provenance signal when their failure modes differ. This Any-survival objective concerns complementary evidence retention, not simultaneous survival of all constituent marks or recovery of all payloads. To our knowledge, this is the first systematic study of neural audio watermark multiplexing, with a scoped benchmark covering five released systems, parallel and sequential baselines, and 14 evaluation conditions. The benchmark shows complementary failure modes, but naive composition does not reliably turn them into system-level robustness under the Any-survival objective. We therefore formulate multiplexing as watermark allocation and study perceptual-adaptive time-frequency multiplexing (PA-TFM), a training-free routing method, and MaskNet, a learned time-domain router for separately trained systems with native detectors. We use the five-system scoped benchmark to characterize multiplexing behavior and the AudioSeal-PerTh pair as a representative heterogeneous pair for adaptive watermark routing. Compared with direct parallel composition, MaskNet improves average TPR@1%FPR from 0.75 to 0.88 and SNR from 15.20 dB to 25.36 dB; compared with PA-TFM, it gives a clear system-level robustness gain under the Any-survival objective while retaining similar high-fidelity behavior. A SpeechTokenizer-aware case study further improves average TPR@1%FPR to 0.91 and SpeechTokenizer robustness from 0.20 to 0.60, showing that channel-adapted constituents can enter the same routing framework.

eess.AS↗

arXiv2026

Geometric Realizations of Capability Spaces of Large Models and Zigzag Persistent Homology

We propose a topological model for capability changes during large-model training. The model represents knowledge states and preregistered capability probes as subcomplexes of a finite regular CW complex. We prove that attaching a single $n$-cell can only create a class in $\mathrm H_n$ or kill a class in $\mathrm H_{n-1}$. With the reliability threshold and the training checkpoint as two parameter axes, capability gains and losses make the spaces along the training axis nonnested. Union or intersection bridges produce zigzag persistence modules, and we distinguish checkpoint, transition, and bridge-sensitive classes. A finite four-stage example is computed by boundary matrices. The framework records algebraic changes under a chosen encoding, and a homological change is called a candidate capability emergence only after robustness tests, null-model comparisons, and independent behavioral validation.

math.AT↗

arXiv2026

FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization

Federated Learning (FL) enables privacy-preserving fine-tuning of Large Language Models (LLMs), yet the massive communication overhead remains a critical bottleneck. Furthermore, applying Low-Rank Adaptation (LoRA) in FL faces a fundamental "aggregation dilemma" between the accurate Sum-of-Products (SoP) and the communication-efficient Product-of-Sums (PoS) implementations. To tackle these challenges, we propose FedFit. First, to significantly reduce communication overhead, we introduce a disjoint shared vector-bank parameterization that reconstructs high-dimensional adapter matrices from two compact and disjoint global vector banks. Second, to address the aggregation dilemma, we devise an alternating optimization schedule. By cycling between decoupled single-bank updates (which allow for accurate aggregation) and joint updates corrected by a Residual Spectral Aggregation mechanism, we resolve the conflict between SoP and PoS. Additionally, we integrate blockwise quantization with client-side error feedback to further compress the transmitted vectors. Furthermore, we establish theoretical convergence guarantees for the proposed algorithm. Extensive experiments on Qwen2.5 models demonstrate that FedFit achieves perplexity performance comparable to standard federated LoRA methods, while providing compression ratios up to 100x higher.

cs.LG↗

arXiv2026

Speech-based Psychological Crisis Assessment using LLMs

Psychological support hotlines provide critical support for individuals experiencing mental health emergencies, yet current assessments largely rely on human operators whose judgments may vary with professional experience and are constrained by limited staffing resources. This paper proposes a large language model (LLM)-based framework for automated crisis level classification, a key indicator that supports many downstream tasks and improves the overall quality of hotline services. To better capture emotional signals in spoken conversations, we introduce a paralinguistic injection method that inserts identified non-verbal emotional cues into speech transcripts, enabling LLM-based reasoning to incorporate critical acoustic nuances. In addition, we propose a reasoning-enhanced training strategy that trains the model to generate diagnostic reasoning chains as an auxiliary task, which serves as a regulariser to improve classification performance. Combined with data augmentation, our final system achieves a macro F1-score of 0.802 and an accuracy of 0.805 on the three-class classification task under 5-fold cross-validation.

cs.CL↗

arXiv2026

Talk2Agent: Benchmarking Voice Interfaces for Text Agents

Large language model (LLM) computer-use agents are typically evaluated with clean written instructions, despite speech being an increasingly popular interface for interacting with such systems. Speech input introduces an additional failure point: transcription errors can alter task-critical entities, constraints, or targets before the agent begins reasoning, while conventional ASR metrics do not directly measure whether the information required for successful execution has been preserved. We introduce Talk2Agent, a benchmark for evaluating how effectively voice interfaces convey human-spoken instructions to LLM-based computer-use agents. Talk2Agent builds human-spoken versions of tasks from WildClawBench and OSWorld and evaluates a range of voice interfaces, including dedicated ASR models, audio-capable LLMs, contextual biasing, and LLM-based ontology repair. Because repeatedly executing long-horizon computer-use tasks is costly and stochastic, we further propose an execution-free, task-conditioned evaluation framework that projects the original task grader onto prompt-addressable intentions and measures how much task-relevant information is retained after the voice interface. On WildClawBench, Talk2Agent's execution-free native projection provides a practical, execution-grounded measure of voice-interface quality, correlating with downstream task completion and improving Pearson correlation by 0.246 over WER/CER on 32 hours of real human speech.

cs.AI↗

arXiv2026

Explainable and Generalisable LLM-based Cognitive Decline Detection with Spontaneous Speech

Alzheimer's disease (AD) and mild cognitive impairment (MCI), which may precede AD, manifest early through subtle linguistic and acoustic alterations. Traditional diagnostics, however, are often resource-intensive and lack scalability for mass screening. To address these challenges, we introduce a novel bilingual speech large language model framework for automated, explainable cognitive screening. Unlike conventional pipelines that rely on error-prone automatic speech recognition, our system directly processes raw speech to learn joint acoustic-semantic representations, preserving critical prosodic cues often lost in transcription. Utilising our newly collected PUTH-AD dataset alongside multiple open-source corpora, we implemented a multi-task learning objective that simultaneously performs cognitive status classification and generates clinician-understandable natural language explanations. Our system achieved the highest average accuracy and AUROC across six dataset/task conditions, comparing three representative baselines. The system demonstrated cross-task transfer to held-out PUTH-AD task subsets, maintaining classification accuracy on an entirely unseen cognitive task without task-specific fine-tuning. Furthermore, clinician evaluation confirms that the generated explanations are both clinically relevant and largely consistent with the underlying speech evidence, supporting their potential utility in clinical interpretation. This study provides a scalable, objective, and explainable framework for speech-based cognitive screening, combining cognitive status classification with natural language explanations that clinicians can assess and verify, bridging the gap between advanced AI and clinical utility.

eess.AS↗

arXiv2026

SALMONN-duo: Adaptive Dual-System Coordination for Full-Duplex Voice Agents

Full-duplex speech large language models (LLMs) enable low-latency, natural voice interaction. However, real-world agents must also use tools and perform deliberative reasoning-operations whose variable latency and computational cost conflict with the stringent timing requirements of real-time conversation. To reconcile these demands, we propose SALMONN-duo, an adaptive dual-system voice agent inspired by dual-process theories of cognition. SALMONN-duo separates real-time interaction from deliberative computation by pairing an always-on, fast-thinking full-duplex speech LLM (system 1) with a powerful asynchronous slow-thinking LLM agent (system 2). Beyond handling real-time interaction, system 1 learns when to answer directly and when to delegate, remaining responsive during backend execution and seamlessly integrating returned information into the ongoing dialogue without exposing tool traces or losing conversational context. Evaluations on single-turn spoken question answering (QA) and multi-turn conversations demonstrate that adaptive delegation substantially improves accuracy on knowledge-intensive and multi-hop reasoning questions, while knowledge-boundary-aware training avoids unnecessary system 2 invocations. On a customized version of $τ$-Voice, SALMONN-duo further demonstrates its ability to complete environment-grounded, policy-constrained tasks through multi-turn interactions in realistic business scenarios. Finally, cost-aware reinforcement learning further enhances the trade-off between task performance and backend usage across the QA and conversation tasks, while improving task success and response safety on $τ$-Voice with an acceptable increase in the delegation rate.

cs.CL↗

arXiv2026

WeaveData: A Multimodal Data Analysis System with Self-Critiquing and Self-Evolving LLM Plans

Multimodal data analysis, which answers questions over relational tables, text, and images, has attracted growing attention in the data management community. Large language models (LLMs) enable such analysis in natural language by generating analysis plans over relational and semantic operators. However, LLM-generated plans are error-prone: a plan may silently compute something other than what was asked, fail during execution, or return a result that misses the question. This paper presents WeaveData, a multimodal data analysis system with self-critiquing and self-evolving LLM plans. First, WeaveData generates a typed logical plan for each question and critiques it step by step before execution, and it checks the executed result against the question afterwards. Second, WeaveData evolves a plan that fails or misses the question: it diagnoses the failure with the actual data, reuses the results that remain valid, and accumulates planning experience for later questions. Third, WeaveData grounds planning in a metadata knowledge graph of all modalities, clarifies ambiguous questions with the user, and backs every model judgment with evidence in an interactive notebook. We demonstrate WeaveData on two public multimodal datasets.

cs.DB↗