arXiv Science⌕ Search

arXiv subjects

Kai Zhu

Publications and source records attributed to Kai Zhu.

At least 19 recordsLinked to original sources
arXiv2026

Joint-Conditioned Stereo Surface Reasoning for Interaction Field Estimation

Predicting hand--object interaction fields requires locating the nearest object-surface point for each hand joint, often from small and partially occluded image regions. We view this task as joint-conditioned surface-endpoint estimation: each joint has its own nearest endpoint, while endpoints from the same hand can draw on shared local surface evidence. This structure motivates Joint-Conditioned Stereo Surface Reasoning (JSSR). A temporal-stereo network jointly predicts 3D joints, a direct interaction field, and per-view endpoint evidence. Calibrated candidate search evaluates endpoint hypotheses using joint-specific image compatibility and cross-view correspondence. A hand-shared candidate support lets joints draw on common surface evidence, and a learned residual gate controls the geometric correction when observations are ambiguous. Our system built on this method ranked third on the SHOW3D Interaction Field Challenge leaderboard.

cs.CV↗
arXiv2026

Adaptive Nesterov Momentum Method for Electrical Impedance Tomography with the Complete Electrode Model

We apply the adaptive Nesterov momentum (ANM) method [30] to electrical impedance tomography under the complete electrode model. The forward problem is formulated in the variational CEM setting, accounting for finite electrode size, contact impedance, insulating gaps, and the mean-free voltage gauge. The resulting nonlinear inverse problem is treated within a unified dual-to-primal framework using three classes of strongly convex structural penalties: an L2-type penalty, an L1-type penalty promoting sparse deviations from a calibrated homogeneous background, and a TV-type penalty favoring approximately piecewise-constant conductivities with sharp interfaces. The TV class is implemented using both smoothed TV and Huber-TV formulations. The data- misfit gradient is computed through CEM adjoint equations and stabilized by Sobolev smoothing. The method is evaluated on the publicly available KIT4 tank measurement data after calibration of the background conductivity and contact impedance from no-object measurements. The experiments include single, multiple, mixed-conductivity, and geometrically challenging phantom configurations. The L2-type penalty generally produces smooth but diffuse reconstructions, whereas the L1-type penalty yields cleaner backgrounds with occasional geometric distortion. The smoothed TV and Huber-TV penalties provide more spatially coherent localization and exhibit similar reconstruction behavior across most tested configurations. These results demonstrate the practical applicability of the adaptive Nesterov framework to measured CEM-EIT data.

math.NA↗
arXiv2026

When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?

Unified multimodal models (UMMs) can perform both understanding and generation, raising a central question: can visual generation improve understanding? Existing evaluations provide mixed evidence, but confound task difficulty, reasoning paradigms, and the closed-loop interaction between generation and understanding. We introduce VGAU-Diag, a fine-grained evaluation framework for vision generation-assisted understanding. It stratifies samples by difficulty, enables unified evaluation of multiple reasoning paradigms, and uses Oracle-Assisted Reference Protocols. Our analysis shows that generated visual aids help on easier instances but become unreliable as reasoning complexity increases. Oracle-assisted diagnosis further reveals that the main bottleneck often lies on the visual-understanding side rather than the visual-generation side, as current UMMs struggle to leverage even faithful visual aids. We also show that effective visual generation should target visual-understanding bottlenecks rather than add more reasoning steps, and identify a three-stage transition from task-irrelevant noise, to misleading plausible guidance, and finally to useful assistance. These findings would be useful to guide the development of better UMMs.

cs.CV↗
arXiv2026

Video = World + Event Stream

We present Wan-Streamer v0.3, which reframes our native-streaming interaction model under a single organizing view: a video is a world plus an event stream. The world is the persistent context in which a video unfolds, including the environment, scene, subjects, ambient acoustic conditions, voice characteristics, and other relatively stable conditions. The event stream is everything that changes over time within that world, including scene or environmental changes, subject behavior, speech, and other sounds. This yields a general-purpose pretraining task over large amounts of real video: given a world and incoming input, predict how the world moves, changes, and responds in real time. The resulting competence can be specialized to a broad family of real-time downstream tasks. We instantiate it on real-time full-duplex audio-visual interaction, where the event stream is the agent's speech together with free-form behavior. Functionally, the model's multimodal understanding process is vision-language-action-like: it maps multimodal user input to language-form speech and behavior actions. Wan-Streamer v0.3 preserves the v0.2 operating point: 640x368 video at 25 FPS, a 160 ms streaming unit, approximately 200 ms model-side response latency, and approximately 550 ms total interaction latency under a 350 ms bidirectional network budget.

cs.CV↗
arXiv2026

Open-AoE: An Open Egocentric Manipulation Dataset and Toolchain for Embodied Learning

Egocentric videos of human manipulation provide scalable supervision for embodied intelligence, yet existing resources rarely combine low-cost continuous capture, manipulation-level structured annotations, and reusable tools for robot learning. We present Open-AoE, an open, community-oriented egocentric manipulation dataset and toolchain spanning the full pipeline from smartphone capture to model training. Its first release contains approximately 2,000 hours of manipulation video collected in natural environments by 500+ contributors using 400+ smartphones. The dataset provides text annotations, MANO-based hand poses, camera trajectories, and temporally localized atomic actions. Open-AoE further includes a data processing pipeline that transforms raw recordings into structured samples through temporal action segmentation, semantic annotation, hand reconstruction, and camera trajectory reconstruction. Meanwhile, we provide a separate downstream toolchain supports visualization, cross-embodiment retargeting, model-specific data conversion, and training recipes for VLA policies, WAMs, and World Models. By integrating scalable capture, structured processing, and downstream adaptation, Open-AoE reduces the barriers to both data contribution and reuse, providing practical open infrastructure for embodied model training, human-to-robot transfer, and world modeling.

cs.RO↗
arXiv2026

Answering Without Referring: How AI Search Rewrites the Web's Economic Bargain

Search engines have long allocated attention on the web by routing users from queries to websites. AI search changes this arrangement because information needs can be resolved inside the intermediary. Using URL-level Comscore U.S. desktop clickstream, we compare ChatGPT and Google information-seeking occasions and exploit ChatGPT Search access expansions to estimate traditional search displacement. ChatGPT produces outbound clicks in only 5.2% of conversation sessions, far below Google's referral ratio. The remaining clicks are not a scaled-down Google stream: they skew toward specialized destinations and away from ad-supported sites. Wider access cuts search use by 9.4%, with search-referral losses largest for informational categories. Our findings identify a central economic shift in digital intermediation: AI search might satisfy information needs inside the intermediary while weakening the referral bargain that has linked search, traffic, and content production on the open web.

cs.CY↗
arXiv2026

Wan-Streamer v0.2: Higher Resolution, Same Latency

We present Wan-Streamer v0.2, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model. v0.2 keeps the v0.1 modeling formulation, but raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS. The higher-resolution stream supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation. To support the larger visual stream without adding user-visible delay, v0.2 keeps the thinker as a single-GPU low-latency path for streaming perception, the short language/state Transformer pass that builds the generation cache, and final decoding. The performer becomes a multi-GPU Ulysses-style context-parallel group for the expensive next-unit latent generation. Each performer rank writes incoming K/V into a pre-sharded local cache. The long high-resolution latent video sequence is split across ranks for denoising and gathered through Ulysses communication, while the much shorter audio latent sequence is generated without sequence sharding. In this split, the thinker's language/state computation reaches the performer only as K/V conditioning, so no separate language sequence has to be communicated inside the performer group. This concentrates additional hardware on visual generation while preserving the compact thinker-performer boundary, keeping total remote interaction latency at approximately 550 ms when a 350 ms bidirectional network budget is included.

cs.CV↗
arXiv2026

Navigating committor landscape of biomolecules with a general pairwise interaction model

Sampling rare conformation transitions between metastable states is a central challenge in atomistic simulations. While the committor function serve as an ideal reaction coordinate for driving enhanced sampling, their high-dimensional inputs and complex functional forms limit the efficacy of standard feedforward neural networks in modeling them. Inspired by recent breakthroughs in biomolecular structure prediction, we propose a novel committor learning framework grounded in the AlphaFold 3 paradigm. By integrating a lightweight, differentiable atom-level embedding with a simplified Pairformer architecture, our method inherently captures intricate dynamical features of diverse biosystems without requiring specialized prior knowledge. We demonstrate the superior expressiveness and accuracy of the proposed framework across multiple atomistic processes. For the folding of the chignolin mini-protein, our model reveals the finer-grained structure of its transition state ensemble (TSE) and a detailed bifurcated reaction mechanism. Furthermore, for calixarene host-guest systems, we develop a unified committor model that elucidates how ligand substituents regulate the ratio between distinct binding pathways, offering new perspectives for structure-based drug design.

physics.comp-ph↗
arXiv2026

Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models

We present Wan-Streamer, a native-streaming, end-to-end interactive foundation model designed from the ground up for real-time, low-latency, full-duplex audio-visual interaction. Wan-Streamer seamlessly models language, audio, and video as both input and output within a single Transformer, where the sequence is represented as interleaved visual, audio, and text input tokens together with visual, audio, and text output tokens, coordinated by block-causal attention for incremental streaming. Unlike cascaded interactive systems that rely on separate VAD, ASR, language, TTS, audio-driven animation, or video-generation modules, Wan-Streamer does not rely on external language, speech, avatar, or video-generation modules: perception, reasoning, generation, response timing, turn management, and cross-modal synchronization are learned jointly within one unified model, reducing pipeline latency and error accumulation. To support natural audio-visual responsiveness, we redesign the entire stack around streamability, including causal encoders, causal decoders, block-causal attention, and low-latency multimodal token scheduling, enabling streaming units as short as 160 ms at 25 fps. Wan-Streamer achieves approximately 200 ms model-side response latency and approximately 550 ms total interaction latency when combined with 350 ms bidirectional network latency, supporting sub-second duplex audio-visual communication. These results position Wan-Streamer as a unified, end-to-end, multimodal interactive foundation model for low-latency streaming interaction.

cs.CV↗
arXiv2026

TaCauchy: An Extensible FEM Framework for Vision-Based Tactile Simulation

Vision-based tactile sensors require high-fidelity simulation for reinforcement learning, yet existing approaches struggle to provide accurate mechanical stress fields within GPU-accelerated robotics platforms. We present TaCauchy, an extensible Finite Element Method (FEM) framework that integrates rigorous physics-based force computation into Isaac Sim. Built on the Unified Incremental Potential Contact (UIPC) solver, TaCauchy directly computes Cauchy stress tensors from hyperelastic constitutive laws and projects them onto contact surfaces to obtain traction forces and pressure distributions, providing mechanical ground truth from first principles rather than empirical estimation. Our framework features automatic mesh generation with geometry-aware adaptive refinement and a modular sensor interface enabling rapid integration of diverse sensors (GelSight Mini, DIGIT, 9DTact) with minimal configuration. Performance benchmarks demonstrate 33.40 FPS for single environments and 555 FPS aggregate throughput across 60 parallel environments, with stress extraction overhead under 1 ms. Physical validation experiments show strong agreement between simulated and real tactile responses across force ranges from 1.2556 N to 4.7332 N, achieving SSIM above 0.93, confirming the framework's capability to provide accurate, physically-grounded force supervision for downstream robotic manipulation tasks.

cs.RO↗
arXiv2026

CUTh-Solver: GPU-Accelerated Sparse Matrix Solver for High-Resolution Thermal Simulation of 3D ICs

Coarse-grained thermal simulation tends to underestimate localized thermal issues, potentially missing critical hotspots. Accurate analysis, therefore, demands fine-grained information, which dramatically increases grid resolution and thus computational workload. Fortunately, the coefficient matrices are often sparse with regular sparsity patterns, offering optimization opportunities. However, existing general-purpose matrix solvers on GPUs rarely exploit these domain-specific properties, thereby encountering bottlenecks in data storage, memory access, parallelism, computational efficiency, and hardware utilization. Therefore, we propose CUTh-Solver, a co-designed GPU-accelerated Preconditioned Conjugate Gradient (PCG)-based sparse solver framework for Symmetric Positive Definite (SPD) systems arising from high-resolution steady-state and transient 3D IC thermal simulation. For data storage, CUTh-Solver condenses the Diagonal (DIA) storage format to remove redundancy. To optimize the memory access, CUTh-Solver employs diagonal-wise SpMV to achieve coalesced memory access. We further observe a critical conflict between parallelism and preconditioning quality and thus adopt a high-parallelism preconditioning strategy. To improve computational efficiency and hardware utilization, we employ an adaptive fine-grained mixed-precision strategy that leverages diverse floating-point units to avoid resource contention, enhancing throughput without compromising numerical stability. Experimental results show that CUTh-Solver achieves up to 25.8x speedup over GPU-accelerated COMSOL Multiphysics 6.4 and over 3x speedup over NVIDIA's native general-purpose libraries (AmgX, cuSPARSE, cuDSS). Ablation studies validate the individual contribution of each optimization. The code is available at: https://github.com/Chenghan-Wang/CUTh-Solver

cs.AR↗
arXiv2026

A Strong Stellar Age-Metallicity Gradient Relation in Nearby Dwarf Galaxies Driven by Stellar Migration and Environmental Quenching

Stellar metallicity gradients ($\nabla[Z/H]$) provide a fossil record of the assembly history of galaxies. We present an analysis of $\nabla[Z/H]$ for 90 nearby low-mass galaxies using VLT/MUSE IFU spectroscopy, spanning stellar masses from $10^{6.5}$ to $10^{10} M_\odot$ (median $\sim 10^{8.5} M_\odot$) and significantly extending the mass coverage of existing IFU surveys into the classical dwarf regime. Our primary finding is a robust negative correlation between $\nabla[Z/H]$ and light-weighted stellar age ($|r|\gtrsim 0.7$) measured out to $\sim$ 2$\times$ effective radius: older dwarf galaxies have steeper (more negative) gradients. This holds regardless of stellar mass, structural compactness, or large-scale environment (group/field), and is strongest in the intermediate-mass regime ($8.2\lesssim\log M_\star/M_\odot\lesssim9.0$). The slope of the age-$\nabla[Z/H]$ relation is close to that in the FIRE-2 simulations, indicating that stellar radial migration driven by feedback-induced potential fluctuations may be fundamental in dwarf evolution. But this apparent consistency is likely coincidental given the simulations' overly efficient feedback and chemical mixing. On the other hand, the H\,\textsc{i} deficiency parameter, an indicator of past environmental stripping, shows a moderate yet highly significant correlation with $\nabla[Z/H]$, second only to stellar age in strength: galaxies with higher H\,\textsc{i} deficiency tend to have more negative gradients, strongly indicating that environment-driven outside-in quenching and the ensuing gradual truncation of metal enrichment re-shape the stellar metallicity distribution. Our analysis suggests that the chemical evolution of dwarf galaxies likely arises from a synergy of feedback-driven dynamical heating and external environmental processing, though only the latter has robust observational support.

astro-ph.GA↗
arXiv2026

SelfTICA: contrastive learning of dynamical representations for rare-event sampling and characterization

Rare events govern many important molecular processes but remain difficult to characterize within accessible simulation timescales. This challenge has motivated machine-learning methods to construct low-dimensional collective variables for specific objectives, including state discrimination, slow-mode identification, and committor-function approximation. Here we introduce SelfTICA, a self-supervised framework that uses contrastive learning on time-lagged configurations to learn a latent representation of the slow modes governing relevant transitions. Once learned, this representation is frozen and reused across downstream tasks, including collective variables construction, enhanced sampling, free-energy estimation, and committor learning. Compared with direct slow-mode optimization, SelfTICA improves training stability and provides collective variables even from limited and exploratory trajectories, accelerating rare-event sampling and enabling accurate free-energy estimation. The dynamical information encoded during pretraining also accelerates committor learning and enables characterization of transition-state regions. These results show that contrastive learning of slow dynamical representations provides a common foundation for rare-event sampling and mechanistic analysis.

physics.comp-ph↗
arXiv2026

Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Models

Reinforcement learning (RL) holds immense promise for enhancing the reasoning capabilities of diffusion large language models (dLLMs). However, progress is fundamentally constrained by a dual misalignment between authentic generation trajectory and the gradient update process: (i) Process-reward misalignment. Sparse, terminal rewards are indiscriminately assigned to all intermediate steps of the generation process, failing to provide discriminative credit assignment. (ii) State-trajectory misalignment. Policy updates are often diverted toward artificial, out-of-trajectory states, squandering gradients on less informative samples. To address these limitations, we introduce Process Aligned Policy Optimization (PAPO), a novel framework that holistically aligns the RL update with the dLLM's generative trajectory via Step-Aware Process Rewards (SPR) that transform sparse terminal rewards into dense, step-wise credit, and Entropy-Guided Historical Re-enactment (EHR) that replays authentic trajectories at high-uncertainty steps. Extensive experiments on four benchmarks demonstrate that PAPO significantly outperforms baselines, achieving gains of 4.5% on GSM8K, 4.8% on MATH500, 42.2% on Countdown and 16.1% on Sudoku.

cs.CL↗
arXiv2026

MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation

Video generation is rapidly evolving from single-shot synthesis to complex multi-shot audio-video (MSAV) narratives to meet real-world demands. However, evaluating such frontier models remains a fundamental challenge. Existing benchmarks are limited in scope and data diversity, and rely on rigid evaluation pipelines, preventing systematic and reliable assessment of modern MSAV models. To bridge these gaps, we introduce MSAVBench, the first comprehensive benchmark and adaptive hybrid evaluation framework for multi-shot audio-video generation. Our benchmark spans four key dimensions, video, audio, shot, and reference, covering diverse task settings, varying shot counts of up to 15, and challenging non-realistic scenarios. Our evaluation framework improves robustness through an adaptive self-correction mechanism for shot segmentation, instance-wise rubrics for subjective metrics, and tool-grounded evidence extraction for complex judgments. Furthermore, MSAVBench achieves high alignment with human judgments, reaching a Spearman rank correlation of 91.5%. Our systematic evaluation of 19 state-of-the-art closed- and open-source models shows that current systems still struggle with director-level control and fine-grained audio-visual synchronization, while modular or agentic generation pipelines offer a promising path toward narrowing the gap between open- and closed-source models. The benchmark data and evaluation code are publicly available at https://github.com/ali-vilab/MSAVBench.

cs.CV↗
arXiv2026

Wan-Image: Pushing the Boundaries of Generative Visual Intelligence

We present Wan-Image, a unified visual generation system explicitly engineered to paradigm-shift image generation models from casual synthesizers into professional-grade productivity tools. While contemporary diffusion models excel at aesthetic generation, they frequently encounter critical bottlenecks in rigorous design workflows that demand absolute controllability, complex typography rendering, and strict identity preservation. To address these challenges, Wan-Image features a natively unified multi-modal architecture by synergizing the cognitive capabilities of large language models with the high-fidelity pixel synthesis of diffusion transformers, which seamlessly translates highly nuanced user intents into precise visual outputs. It is fundamentally powered by large-scale multi-modal data scaling, a systematic fine-grained annotation engine, and curated reinforcement learning data to surpass basic instruction following and unlock expert-level professional capabilities. These include ultra-long complex text rendering, hyper-diverse portrait generation, palette-guided generation, multi-subject identity preservation, coherent sequential visual generation, precise multi-modal interactive editing, native alpha-channel generation, and high-efficiency 4K synthesis. Across diverse human evaluations, Wan-Image exceeds Seedream 5.0 Lite and GPT Image 1.5 in overall performance, reaching parity with Nano Banana Pro in challenging tasks. Ultimately, Wan-Image revolutionizes visual content creation across e-commerce, entertainment, education, and personal productivity, redefining the boundaries of professional visual synthesis.

cs.CV↗
arXiv2026

The Role of Baryonic and Dark Matter in Bar Kinematics

Simulations predict that bars in galaxies should slow down over time. This is often attributed to the exchange of angular momentum between the bar and other regions of the galaxy, such as the outer disc and dark matter halo, which implies that galaxies with a more massive halo or disc should be able to slow down the bar more efficiently. However, observational evidence for this process has been limited. In this work, we provide observational support for the slowing down of bars as predicted by simulations. We combine bar kinematics measurements obtained with the Tremaine-Weinberg method and host galaxy mass estimates derived from Jeans anisotropic models for a sample of 30 galaxies from the MaNGA survey. We find a statistically significant anti-correlation (>4sigma) between the bar pattern speed and both the stellar and total dynamical mass, which suggests that the slowest bars reside in the most massive galaxies. However, while the slope of the best-fit line between the pattern speed and dark matter mass is negative, it is not statistically significant (2.43sigma). We also find that bars with lower pattern speeds have more extended NFW dark matter profiles with lower central densities. Additionally, we find statistically significant correlations (>3sigma) between the corotation radius and the stellar mass, dark matter mass, and total dynamical mass. Finally, we find no significant correlations that involve the dark matter fraction or R, likely due to the inherent challenges associated with measuring these specific parameters accurately.

astro-ph.GA↗
arXiv2026

Wan-Weaver: Interleaved Multi-modal Generation via Decoupled Training

Recent unified models have made unprecedented progress in both understanding and generation. However, while most of them accept multi-modal inputs, they typically produce only single-modality outputs. This challenge of producing interleaved content is mainly due to training data scarcity and the difficulty of modeling long-range cross-modal context. To address this issue, we decompose interleaved generation into textual planning and visual consistency modeling, and introduce a framework consisting of a planner and a visualizer. The planner produces dense textual descriptions for visual content, while the visualizer synthesizes images accordingly. Under this guidance, we construct large-scale textual-proxy interleaved data (where visual content is represented in text) to train the planner, and curate reference-guided image data to train the visualizer. These designs give rise to Wan-Weaver, which exhibits emergent interleaved generation ability with long-range textual coherence and visual consistency. Meanwhile, the integration of diverse understanding and generation data into planner training enables Wan-Weaver to achieve robust task reasoning and generation proficiency. To assess the model's capability in interleaved generation, we further construct a benchmark that spans a wide range of use cases across multiple dimensions. Extensive experiments demonstrate that, even without access to any real interleaved data, Wan-Weaver achieves superior performance over existing methods.

cs.CV↗