arXiv subjects

Wenbo Zhao

Publications and source records attributed to Wenbo Zhao.

At least 19 recordsLinked to original sources

arXiv2026

ViP-Rig: Visual-Prompted Controllable Rigging

Rigging is inherently task-dependent because the same mesh may require different skeletons and deformation behaviors across animation tasks. In practice, artists often inspect an initial rig and repeatedly edit its skeletal structure and deformation behavior to meet specific animation requirements. Existing automatic methods primarily generate a plausible rig from geometry, offering limited explicit control over the resulting skeleton and deformation behavior. In this work, we present ViP-Rig, a visual-prompted framework that supports both prompt-first rigging and result-guided editing by injecting features extracted from user-drawn or edited 2D skeletal and rigidity prompts into frozen pretrained backbones. Specifically, ViP-Rig consists of two stages, Skeleton Generation and Skinning Prediction. In the first stage, the skeletal sketch is processed by the Dense-to-Compact Visual Prompt Encoding to produce compact, fixed-length conditioning tokens. The resulting tokens are injected into a frozen pretrained autoregressive generator through gated adapters to control joint placement and branching structure while preserving the generator's geometric prior. In the second stage, the rigidity map is processed using the same visual encoding design, while the pretrained skinning backbone remains frozen. The resulting tokens are symmetrically injected into the point and joint streams to modulate point-joint compatibility and the resulting skinning weights. Experiments on Articulation-XL2.0 and zero-shot evaluation on ModelsResource show that ViP-Rig more accurately recovers target skeletons and skinning weights than geometry-conditioned baselines under prompt-guided evaluation. Qualitative results further demonstrate explicit and localized control in both prompt-first rigging and result-guided editing.

cs.CV↗

arXiv2026

Cloud-top infrared observations reveal the four-dimensional precipitation structure

Accurate four-dimensional (4D) precipitation information is essential for understanding the Earth's energy and water cycles, yet remains observationally unresolved at global scales. Conventional theory holds that geostationary infrared observations primarily sense cloud-top properties, with limited sensitivity to sub-cloud precipitation. Here we show that cloud-top infrared measurements nevertheless encode sufficient information to recover the four-dimensional structure of precipitation, revealing a previously unexploited observability of sub-cloud processes. We introduce a physically constrained deep learning framework, 4DPrecipNet, in which a moisture-first constraint requires the latent representation to recover precipitable water vapour, anchoring the model in thermodynamic consistency. By integrating multi-channel infrared radiances with these constraints and radar-derived precipitation profiles, we reconstruct the vertical and temporal evolution of precipitation systems from geostationary orbit. The framework captures deep convective structures and their evolution, with robust performance across large samples and independent radar comparisons. These results demonstrate that sub-cloud precipitation is physically encoded in cloud-top infrared observations, establishing a new pathway for continuous global monitoring of precipitation structure.

cs.CV↗

arXiv2026

Crystal structure prediction with nuclear quantum and finite-temperature effects via deep free energy learning

Accurate crystal structure prediction (CSP) requires accounting for finite-temperature and nuclear quantum effects, yet first-principles evaluation of the free energy surface (FES) remains prohibitive for high-throughput searches. We observe that the self-consistent harmonic approximation (SCHA) FES, as a function of nuclear centroid positions, shares the same mathematical structure as a potential-energy surface and can therefore be directly learned by a deep neural network potential. The resulting deep free energy (DF) model, constructed via a two-level concurrent-learning workflow, evaluates free energies, forces, and stresses in a single forward pass. Applied to the La-Sc-H system at 200 GPa and 300 K, DF-based CSP reproduces the stability of the experimentally observed LaH10 and LaSc2H24, and discovers an unreported thermodynamically stable clathrate hydride: P4/mmm LaScH8. Benchmarked on the LaH10 system, the DF model achieves a 1.72*10^6-fold cost reduction relative to DFT-level SSCHA. The DF framework provides a scalable route for incorporating finite-temperature and nuclear quantum effects into high-throughput crystal structure prediction.

cond-mat.mtrl-sci↗

arXiv2026

Isotropic Superconductivity in Room-temperature Superconductor LaSc$_{2}$H$_{24}$

The discovery of LaSc$_{2}$H$_{24}$ represents a milestone in the quest for room-temperature superconductivity, yet the microscopic mechanism underlying its superior performance remains unclear. Through a comprehensive revisit of theoretical calculations, we uncover a pivotal transition from the anisotropic two-gap superconductivity of LaH$_{10}$ to the isotropic single-gap superconductivity in LaSc$_{2}$H$_{24}$ upon the introduction of scandium, thereby enhancing the superconducting critical temperature ($T_\mathrm{c}$). This enhancement is rooted in a critical dual role of Sc $3d$ electrons: i) the Sc-derived Jahn-Teller effect promotes hydrogen metallization via the elongation of specific interlayer H-H bonds and enhances electron-phonon coupling (EPC) through the softening of associated phonon modes; ii) Sc $3d$ electrons reconstruct the electronic structure into an MgB$_{2}$-like configuration, generating novel Sc-H-Sc $\sigma$- and $\pi$-bonding states with EPC strengths comparable to LaH$_{10}$. Crucially, the pronounced hybridization between Sc and the hydrogen cages effectively unifies these two contributions on the Fermi surface. This Sc-induced gap unification bridges the high-EPC H-H states with widespread Sc-H states, establishing an isotropic single-gap nature with a large overall EPC strength. Our findings identify this Sc-induced gap unification as the fundamental mechanism for achieving room-temperature superconductivity in LaSc$_{2}$H$_{24}$, offering a theoretical blueprint for the future design of superior superconducting hydrides.

cond-mat.supr-con↗

arXiv2025

Revisiting Phase Stability and Superconductivity in Ca-H Superhydrides with Anharmonic Effects

The prediction of superconductivity above 200 K in CaH$_6$ revolutionized research on hydrogen-rich superconductors, and subsequent experiments have verified this prediction, while unidentified peaks in XRD and the decrease in superconducting temperature upon decompression indicate that unresolved issues remain. In this work, we reconstructed the accurate temperature-pressure phase diagram of the Ca-H system and determined the stability ranges of its candidate superconducting phases by considering anharmonic effects. Our results demonstrate that type-I clathrate Ca$_8$H$_{46-delta}$ structures become thermodynamically stable at 0 K when anharmonic effects are considered. Notably, we found that the previously predicted CaH$_6$ phase achieves stability above 500 K, underscoring the significant role of temperature and anharmonic effects in stabilizing this intriguing high-pressure phase. Our findings offer insights into the structure and superconducting mechanisms of hydrides.

cond-mat.supr-con↗

arXiv2025

Rethinking Autoregressive Models for Lossless Image Compression via Hierarchical Parallelism and Progressive Adaptation

Autoregressive (AR) models, the theoretical performance benchmark for learned lossless image compression, are often dismissed as impractical due to prohibitive computational cost. This work re-thinks this paradigm, introducing a framework built on hierarchical parallelism and progressive adaptation that re-establishes pure autoregression as a top-performing and practical solution. Our approach is embodied in the Hierarchical Parallel Autoregressive ConvNet (HPAC), an ultra-lightweight pre-trained model using a hierarchical factorized structure and content-aware convolutional gating to efficiently capture spatial dependencies. We introduce two key optimizations for practicality: Cache-then-Select Inference (CSI), which accelerates coding by eliminating redundant computations, and Adaptive Focus Coding (AFC), which efficiently extends the framework to high bit-depth images. Building on this efficient foundation, our progressive adaptation strategy is realized by Spatially-Aware Rate-Guided Progressive Fine-tuning (SARP-FT). This instance-level strategy fine-tunes the model for each test image by optimizing low-rank adapters on progressively larger, spatially-continuous regions selected via estimated information density. Experiments on diverse datasets (natural, satellite, medical) validate that our method achieves new state-of-the-art compression. Notably, our approach sets a new benchmark in learned lossless compression, showing a carefully designed AR framework can offer significant gains over existing methods with a small parameter count and competitive coding speeds.

cs.CV↗

arXiv2025

Topological Catenation-induced Pore Size in 2D Olympic Network

Assemblies of ring DNA and proteins, such as kinetoplast DNA (kDNA) and the chainmail-like network of the HK97 bacteriophage capsid, form "Olympic" topologies that govern the dynamics of gene-regulatory proteins, cellular metabolites, and viral genomes. Yet the pore sizes that control transport and diffusion in these topologically linked networks remain poorly quantified. Here, using coarse-grained simulations of idealized square (SQR) and hexagonal (HEX) lattice catenated two-dimensional Olympic networks, we measure pore size as the radius of the largest rigid tracer that can pass through an aperture and delineate how backbone bending stiffness and topological tension regulate this structure. We identify two competing regulators: (i) ring entropic elasticity, which enlarges pores at low stiffness, and (ii) ring rotation angle, which reduces pore size at high stiffness. Their competition yields bimodal pore size distributions over a defined parameter range. Network geometry further modulates spatial correlations in pore size and the pore size autocorrelation time, predicting distinct propagation of perturbations in isostatic (SQR) versus hypostatic (HEX) architectures. These results provide testable predictions for enzyme diffusion kinetics and cargo release in natural or synthetic catenane DNA networks and mechanically interlocked capsid proteins, and offer a theoretical framework for interpreting FRAP (fluorescence recovery after photobleaching) and SMT (single-molecule tracking) measurements in topologically catenated biopolymer networks.

cond-mat.soft↗

arXiv2025

VDEGaussian: Video Diffusion Enhanced 4D Gaussian Splatting for Dynamic Urban Scenes Modeling

Dynamic urban scene modeling is a rapidly evolving area with broad applications. While current approaches leveraging neural radiance fields or Gaussian Splatting have achieved fine-grained reconstruction and high-fidelity novel view synthesis, they still face significant limitations. These often stem from a dependence on pre-calibrated object tracks or difficulties in accurately modeling fast-moving objects from undersampled capture, particularly due to challenges in handling temporal discontinuities. To overcome these issues, we propose a novel video diffusion-enhanced 4D Gaussian Splatting framework. Our key insight is to distill robust, temporally consistent priors from a test-time adapted video diffusion model. To ensure precise pose alignment and effective integration of this denoised content, we introduce two core innovations: a joint timestamp optimization strategy that refines interpolated frame poses, and an uncertainty distillation method that adaptively extracts target content while preserving well-reconstructed regions. Extensive experiments demonstrate that our method significantly enhances dynamic modeling, especially for fast-moving objects, achieving an approximate PSNR gain of 2 dB for novel view synthesis over baseline approaches.

cs.CV↗

arXiv2025

The Fourth Monocular Depth Estimation Challenge

This paper presents the results of the fourth edition of the Monocular Depth Estimation Challenge (MDEC), which focuses on zero-shot generalization to the SYNS-Patches benchmark, a dataset featuring challenging environments in both natural and indoor settings. In this edition, we revised the evaluation protocol to use least-squares alignment with two degrees of freedom to support disparity and affine-invariant predictions. We also revised the baselines and included popular off-the-shelf methods: Depth Anything v2 and Marigold. The challenge received a total of 24 submissions that outperformed the baselines on the test set; 10 of these included a report describing their approach, with most leading methods relying on affine-invariant predictions. The challenge winners improved the 3D F-Score over the previous edition's best result, raising it from 22.58% to 23.05%.

cs.CV↗

arXiv2025

FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion

Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability stemming from two factors: 1) limited annotated image-event-depth datasets causing insufficient cross-modal supervision, and 2) inherent frequency mismatches between static images and dynamic event streams with distinct spatiotemporal patterns, leading to ineffective feature fusion. To address this dual challenge, we propose Frequency-decoupled Unified Self-supervised Encoder (FUSE) with two synergistic components: The Parameter-efficient Self-supervised Transfer (PST) establishes cross-modal knowledge transfer through latent space alignment with image foundation models, effectively mitigating data scarcity by enabling joint encoding without depth ground truth. Complementing this, we propose the Frequency-Decoupled Fusion module (FreDFuse) to explicitly decouple high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches through physics-aware fusion. This combined approach enables FUSE to construct a universal image-event encoder that only requires lightweight decoder adaptation for target datasets. Extensive experiments demonstrate state-of-the-art performance with 14% and 24.9% improvements in Abs .Rel on MVSEC and DENSE datasets. The framework exhibits remarkable zero-shot adaptability to challenging scenarios including extreme lighting and motion blur, significantly advancing real-world deployment capabilities. The source code for our method is publicly available at: https://github.com/sunpihai-up/FUSE

cs.CV↗

arXiv2025

Solid-state Synapse Based on Magnetoelectrically Coupled Memristor

Brain-inspired computing architectures attempt to emulate the computations performed in the neurons and the synapses in human brain. Memristors with continuously tunable resistances are ideal building blocks for artificial synapses. Through investigating the memristor behaviors in a La0.7Sr0.3MnO3/BaTiO3/La0.7Sr0.3MnO3 multiferroic tunnel junction, it was found that the ferroelectric domain dynamics characteristics are influenced by the relative magnetization alignment of the electrodes, and the interfacial spin polarization is manipulated continuously by ferroelectric domain reversal, enriching our understanding of the magnetoelectric coupling fundamentally. This creates a functionality that not only the resistance of the memristor but also the synaptic plasticity form can be further manipulated, as demonstrated by the spike-timing-dependent plasticity investigations. Density functional theory calculations are carried out to describe the obtained magnetoelectric coupling, which is probably related to the Mn-Ti intermixing at the interfaces. The multiple and controllable plasticity characteristic in a single artificial synapse, to resemble the synaptic morphological alteration property in a biological synapse, will be conducive to the development of artificial intelligence.

cond-mat.mtrl-sci↗

arXiv2025

The Impact of Ionic Anharmonicity on Superconductivity in Metal-Stuffed B-C Clathrates

Metal-stuffed B$-$C compounds with sodalite clathrate structure have captured increasing attention due to their predicted exceptional superconductivity above liquid nitrogen temperature at ambient pressure. However, by neglecting the quantum lattice anharmonicity, the existing studies may result in an incomplete understanding of such a lightweight system. Here, using state-of-the-art ab initio methods incorporating quantum effects and machine learning potentials, we revisit the properties of a series of $XY$$\text{B}_{6}\text{C}_{6}$ clathrates where $X$ and $Y$ are metals. Our findings show that ionic quantum and anharmonic effects can harden the $E_g$ and $E_u$ vibrational modes, enabling the dynamical stability of 15 materials previously considered unstable in the harmonic approximation, including materials with previously unreported ($XY$)$^{1+}$ state, which is demonstrated here to be crucial to reach high critical temperatures. Further calculations based on the anisotropic Migdal-Eliashberg equation demonstrate that the $T_\text{c}$ values for KRb$\text{B}_{6}\text{C}_{6}$ and Rb$\text{B}_{3}\text{C}_{3}$ among these stabilized compounds are 102 and 115 K at 0 and 15 GPa, respectively, both being higher than $T_\text{c}$ of 92 K of KPb$\text{B}_{6}\text{C}_{6}$ at the anharmonic level. These record-high $T_\text{c}$ values, surpassing liquid nitrogen temperatures, emphasize the importance of anharmonic effects in stabilizing B-C clathrates with large electron-phonon coupling strength and advancing the search for high-$T_\text{c}$ superconductivity at (near) ambient pressure.

cond-mat.supr-con↗

arXiv2024

Adaptive Stereo Depth Estimation with Multi-Spectral Images Across All Lighting Conditions

Depth estimation under adverse conditions remains a significant challenge. Recently, multi-spectral depth estimation, which integrates both visible light and thermal images, has shown promise in addressing this issue. However, existing algorithms struggle with precise pixel-level feature matching, limiting their ability to fully exploit geometric constraints across different spectra. To address this, we propose a novel framework incorporating stereo depth estimation to enforce accurate geometric constraints. In particular, we treat the visible light and thermal images as a stereo pair and utilize a Cross-modal Feature Matching (CFM) Module to construct a cost volume for pixel-level matching. To mitigate the effects of poor lighting on stereo matching, we introduce Degradation Masking, which leverages robust monocular thermal depth estimation in degraded regions. Our method achieves state-of-the-art (SOTA) performance on the Multi-Spectral Stereo (MS2) dataset, with qualitative evaluations demonstrating high-quality depth maps under varying lighting conditions.

cs.CV↗

arXiv2024

MCGS: Multiview Consistency Enhancement for Sparse-View 3D Gaussian Radiance Fields

Radiance fields represented by 3D Gaussians excel at synthesizing novel views, offering both high training efficiency and fast rendering. However, with sparse input views, the lack of multi-view consistency constraints results in poorly initialized Gaussians and unreliable heuristics for optimization, leading to suboptimal performance. Existing methods often incorporate depth priors from dense estimation networks but overlook the inherent multi-view consistency in input images. Additionally, they rely on dense initialization, which limits the efficiency of scene representation. To overcome these challenges, we propose a view synthesis framework based on 3D Gaussian Splatting, named MCGS, enabling photorealistic scene reconstruction from sparse views. The key innovations of MCGS in enhancing multi-view consistency are as follows: i) We leverage matching priors from a sparse matcher to initialize Gaussians primarily on textured regions, while low-texture areas are populated with randomly distributed Gaussians. This yields a compact yet sufficient set of initial Gaussians. ii) We propose a multi-view consistency-guided progressive pruning strategy to dynamically eliminate inconsistent Gaussians. This approach confines their optimization to a consistency-constrained space, which ensures robust and coherent scene reconstruction. These strategies enhance robustness to sparse views, accelerate rendering, and reduce memory consumption, making MCGS a practical framework for 3D Gaussian Splatting.

cs.CV↗

arXiv2024

PVContext: Hybrid Context Model for Point Cloud Compression

Efficient storage of large-scale point cloud data has become increasingly challenging due to advancements in scanning technology. Recent deep learning techniques have revolutionized this field; However, most existing approaches rely on single-modality contexts, such as octree nodes or voxel occupancy, limiting their ability to capture information across large regions. In this paper, we propose PVContext, a hybrid context model for effective octree-based point cloud compression. PVContext comprises two components with distinct modalities: the Voxel Context, which accurately represents local geometric information using voxels, and the Point Context, which efficiently preserves global shape information from point clouds. By integrating these two contexts, we retain detailed information across large areas while controlling the context size. The combined context is then fed into a deep entropy model to accurately predict occupancy. Experimental results demonstrate that, compared to G-PCC, our method reduces the bitrate by 37.95\% on SemanticKITTI LiDAR point clouds and by 48.98\% and 36.36\% on dense object point clouds from MPEG 8i and MVUB, respectively.

cs.CV↗

arXiv2024

CUSIDE-T: Chunking, Simulating Future and Decoding for Transducer based Streaming ASR

Streaming automatic speech recognition (ASR) is very important for many real-world ASR applications. However, a notable challenge for streaming ASR systems lies in balancing operational performance against latency constraint. Recently, a method of chunking, simulating future context and decoding, called CUSIDE, has been proposed for connectionist temporal classification (CTC) based streaming ASR, which obtains a good balance between reduced latency and high recognition accuracy. In this paper, we present CUSIDE-T, which successfully adapts the CUSIDE method over the recurrent neural network transducer (RNN-T) ASR architecture, instead of being based on the CTC architecture. We also incorporate language model rescoring in CUSIDE-T to further enhance accuracy, while only bringing a small additional latency. Extensive experiments are conducted over the AISHELL-1, WenetSpeech and SpeechIO datasets, comparing CUSIDE-T and U2++ (both based on RNN-T). U2++ is an existing counterpart of chunk based streaming ASR method. It is shown that CUSIDE-T achieves superior accuracy performance for streaming ASR, with equal settings of latency.

cs.SD↗

arXiv2024

Spatial Annealing for Efficient Few-shot Neural Rendering

Neural Radiance Fields (NeRF) with hybrid representations have shown impressive capabilities for novel view synthesis, delivering high efficiency. Nonetheless, their performance significantly drops with sparse input views. Various regularization strategies have been devised to address these challenges. However, these strategies either require additional rendering costs or involve complex pipeline designs, leading to a loss of training efficiency. Although FreeNeRF has introduced an efficient frequency annealing strategy, its operation on frequency positional encoding is incompatible with the efficient hybrid representations. In this paper, we introduce an accurate and efficient few-shot neural rendering method named \textbf{S}patial \textbf{A}nnealing regularized \textbf{NeRF} (\textbf{SANeRF}), which adopts the pre-filtering design of a hybrid representation. We initially establish the analytical formulation of the frequency band limit for a hybrid architecture by deducing its filtering process. Based on this analysis, we propose a universal form of frequency annealing in the spatial domain, which can be implemented by modulating the sampling kernel to exponentially shrink from an initial one with a narrow grid tangent kernel spectrum. This methodology is crucial for stabilizing the early stages of the training phase and significantly contributes to enhancing the subsequent process of detail refinement. Our extensive experiments reveal that, by adding merely one line of code, SANeRF delivers superior rendering quality and much faster reconstruction speed compared to current few-shot neural rendering methods. Notably, SANeRF outperforms FreeNeRF on the Blender dataset, achieving 700$\times$ faster reconstruction speed.

cs.CV↗

arXiv2024

Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision

There exist three approaches for multilingual and crosslingual automatic speech recognition (MCL-ASR) - supervised pretraining with phonetic or graphemic transcription, and self-supervised pretraining. We find that pretraining with phonetic supervision has been underappreciated so far for MCL-ASR, while conceptually it is more advantageous for information sharing between different languages. This paper explores the approach of pretraining with weakly phonetic supervision towards data-efficient MCL-ASR, which is called Whistle. We relax the requirement of gold-standard human-validated phonetic transcripts, and obtain International Phonetic Alphabet (IPA) based transcription by leveraging the LanguageNet grapheme-to-phoneme (G2P) models. We construct a common experimental setup based on the CommonVoice dataset, called CV-Lang10, with 10 seen languages and 2 unseen languages. A set of experiments are conducted on CV-Lang10 to compare, as fair as possible, the three approaches under the common setup for MCL-ASR. Experiments demonstrate the advantages of phoneme-based models (Whistle) for MCL-ASR, in terms of speech recognition for seen languages, crosslingual performance for unseen languages with different amounts of few-shot data, overcoming catastrophic forgetting, and training efficiency. It is found that when training data is more limited, phoneme supervision can achieve better results compared to subword supervision and self-supervision, thereby providing higher data-efficiency. To support reproducibility and promote future research along this direction, we release the code, models and data for the entire pipeline of Whistle at https://github.com/thu-spmi/CAT/tree/master/egs/cv-lang10.

cs.SD↗