arXiv Science⌕ Search

arXiv subjects

Hongbo Zhao

Publications and source records attributed to Hongbo Zhao.

At least 19 recordsLinked to original sources
arXiv2026

CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Models

As vision language models are increasingly deployed in clinical diagnosis, understanding how they internally resolve competing visual and textual signals becomes a safety imperative. Existing mechanistic analyses remain confined to unimodal text and offer no explanation for why a single misleading sentence can override a correct image based diagnosis, or why a model commits to a confident answer despite insufficient visual evidence. We find that these two safety risks, arbitration failure where textual context overrides visual grounding and brake failure where the model commits without adequate evidence, are mediated by spatially disjoint attention head populations: arbitration heads form a mid-to-deep wideband reflecting cross-layer evidence competition, while brake heads concentrate in a narrow middle-to-late layer band that regulates evidence sufficiency and abstention behavior. To ground these observations in causal circuitry, we introduce CRAFT, which localizes each failure mode to a minimal causal head set via dual criteria and verifies necessity and sufficiency through temporal probes and Tuned Lens trajectory analysis. Excising arbitration heads sharply reduces conflict following with negligible degradation on clean inputs, while excising brake heads restores appropriate abstention under degraded visual evidence. The two interventions target spatially disjoint head sets and produce distinct corrective effects, underscoring the mechanistic separability of the failure modes. Experiments across multiple medical VQA benchmarks and VLM architectures validate both the localization and interventions, demonstrating that the identified heads causally drive each failure mode and that targeted modulation generalises without retraining. The code is available at https://github.com/zhcz328/CRAFT.

cs.CV↗
arXiv2026

Supercoiling Drives DNA-Protein Co-Condensation

The co-condensation of DNA and proteins plays an important role in genome compaction, including heterochromatin formation and chromosome segregation. Meanwhile, transcription and replication continuously generate DNA supercoiling, yet how the resulting torsional stress affects co-condensation remains poorly understood. Here, by coupling DNA elasticity to a protein concentration field, we show that supercoiling acts as a topological catalyst for co-condensation through a mutually reinforcing interplay with protein wetting. Specifically, protein wetting facilitates twist-to-writhe conversion, generating highly writhed DNA conformations that in turn promote capillary collapse into a protein-rich droplet. In the absence of supercoiling, DNA remains kinetically trapped in metastable extended states within our model, whereas sufficiently strong supercoiling removes the nucleation barrier and enables spontaneous co-condensation. These results establish topology as a distinct control parameter for biomolecular condensation and provide a mechanical framework for how active genomic processes can dynamically regulate large-scale genome organization.

physics.bio-ph↗
arXiv2026

Binaural Audio-Visual Instance Segmentation

Audio-visual segmentation (AVS) aims to segment sounding objects at the pixel level by integrating auditory and visual cues. However, existing methods are predominantly developed under the monaural setting and primarily rely on cross-modal semantic correspondence, which limits their ability to distinguish visually similar instances of the same semantic class. In contrast, humans naturally exploit binaural hearing, where interaural differences and direction-dependent acoustic filtering introduced by the head and pinnae provide physically grounded spatial cues for accurate sound source localization. Motivated by this observation, we introduce binaural audio-visual instance segmentation (BiAVIS), a new task that leverages synchronized binaural audio and video frames to segment sounding instances. To advance research on this task, we establish two benchmarks by manually annotating an existing binaural audio-visual dataset and collecting a new real-world dataset, BiAVIS-Bench, in more challenging and diverse scenarios. The proposed BiAVIS model (BiAVISM) features an audio-only sound source localization network that directly projects binaural spatial cues onto the image plane to learn spatial and semantic priors for sounding instances. A query-level audio-visual fusion strategy is subsequently introduced to inject these informative priors into the instance segmentation decoder. Extensive experiments conducted on the two proposed benchmarks demonstrate the superior performance of the BiAVISM over previous monaural AVS methods, especially in resolving instance-level intra-class ambiguity. On the more challenging BiAVIS-Bench, BiAVISM achieves 26.65 mAP and 45.90 FSLA, corresponding to relative improvements of approximately 35.35% over AVISM in mean average precision (mAP) and 11.62% over ACVIS in frame-level sound localization accuracy (FSLA).

cs.CV↗
arXiv2026

Directional and Quasi-Omnidirectional Indoor Channel Measurements at Upper 6 GHz

This paper presents wideband indoor channel measurements and measurement-based statistical modeling of channel parameters in the upper 6 GHz (U6G) band using directional and quasi-omnidirectional antenna pairs. Measurements span 6.425-7.125 GHz at 32 matched positions per configuration in an open hall and an L-shaped corridor. System calibration and frequency-dependent peak-gain normalization retain antenna radiation-pattern weighting. Close-in and floating-intercept models are fitted to line-of-sight (LOS) and non-LOS (NLOS) path loss, respectively. Shadow-fading residuals, RMS delay spread (DS), and equivalent K factors derived from power delay profiles are characterized. LOS exponents range from 1.98 to 2.14. Despite similar attenuation slopes, directional links exhibit smaller LOS residuals and DS and higher equivalent K factors. Hall DS medians are 1.78 and 36.52 ns for directional and quasi-omnidirectional links. Corridor NLOS slopes are -1.12 and 2.54, respectively, while DS ordering varies by position. These results characterize antenna-dependent parameters for U6G indoor channels.

cs.ET↗
arXiv2026

Spectrality and eigen sets of infinite convolutions and random measures generated by admissible pairs

In this paper, we construct a class of random measures $μ^{\mathbf{n}}$ by infinite convolutions. Given admissible pairs $\{(N_{k}, B_{k})\}_{k=1}^{m}$ and a sequence $\bn=\{n_{k}\}_{k=1}^{\infty}$ of positive integers, for every $\bw\in Ω$, we write $μ^{\mathbf{n}}(\bw) = δ_{N_{ω_{1}}^{-n_{1}}B_{ω_{1}}} * δ_{N_{ω_{1}}^{-n_{1}}N_{ω_{2}}^{-n_{2}}B_{ω_{2}}} * \cdots$. First, we show that the mapping $μ^{\mathbf{n}}: (\bw, B) \mapsto μ^{\mathbf{n}}(\bw)(B)$ is a random measure. Next, we introduce the notion of a $t$-equi-positive family, and use it to obtain a general sufficient condition under which an infinite convolution is a spectral measure and possesses a specified set of spectral eigenvalues. We then extend the concepts of spectrality and spectral eigenvalues to random measures, and show that, under the assumption that the corresponding standard infinite convolution is non-degenerate for $\mathbb{P}$-a.e.\ $\boldsymbolω\inΩ$, the measures $μ^{\mathbf{n}}$ are spectral random measures for $\mathbb{P}$-a.e.\ $\boldsymbolω$, admitting the spectral eigen set \[ \mathcal{E}_m=\{t\in\mathbb{N}_+:\gcd(t,N_k)=1,\,1\le k\le m\}. \] Moreover, for each such $t$, there exist uncountably many spectra $Λ_{\boldsymbolω}\subset\mathbb{Z}$ with $tΛ_{\boldsymbolω}$ also a spectrum of $μ^{\mathbf{n}}(\boldsymbolω)$. Finally, for the important case where each digit set $B_k$ is a consecutive set $\{0,1,\dots,b_k-1\}$, we completely characterise the positive integer spectral eigenvalues of $μ^{\mathbf{n}}(\boldsymbolω)$, proving that they are exactly the integers coprime to every $b_k$.

math.FA↗
arXiv2026

Hierarchical Physics-Embedded Learning for Partially Known Spatiotemporal Dynamics

Partial physical knowledge--governing structures known, constitutive relations or their combinations not--pervades spatiotemporal systems. Existing scientific machine learning paradigms learn evolution largely from data, impose equations as soft constraints, or hard-code physical terms into network updates; none exploits knowledge of this form. Here we introduce the hierarchical physics-embedded adaptive Fourier neural operator, encoding such knowledge as computational architecture rather than penalizing or appending it: a first level learns or embeds fundamental physical expressions as intermediate representations, and a second level learns or embeds their governing combination, with adaptive Fourier layers capturing nonlocal, high-order couplings at each level. We prove a hierarchical error decomposition--embedding known components removes or shrinks their terms, and a parameter-complexity advantage: when the hierarchy aligns with the compositional structure of the dynamics, the number of learnable Fourier parameters sufficient for a prescribed accuracy grows strictly more slowly than for a single-level operator. Across canonical phase-field systems and experimental hydrofoil wake data, our method reduces long horizon extrapolation errors by up to ~70% relative to state-of-the-art physics encoded and neural operator baselines, while preserving physically meaningful morphology, energetic consistency, and spectral structure, and maintaining robust performance under sparse and noisy observations. The separated intermediate representations further enable symbolic recovery of unknown constitutive relations in partially specified PDEs. These results establish hierarchical physics embedding as a theoretically grounded route to prediction and discovery when governing laws are neither fully known nor absent, but partially known and compositionally organized.

cs.LG↗
arXiv2026

Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training

Continual post-training (CPT) is a popular and effective technique for adapting foundation models like multimodal large language models to ever-evolving downstream tasks. While existing research primarily focuses on methods like data replay, model expansion, or parameter regularization, the fundamental role of the learning paradigm remains largely unexplored. This paper presents a comparative analysis of two core post-training paradigms: supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT), investigating their respective impacts on knowledge retention during CPT. Our experiments are conducted across multiple multimodal tasks, utilizing Qwen2.5-VL-7B-Instruct as the base model. The investigation yields two significant findings: (1) When continuously learning on downstream tasks, SFT leads to catastrophic forgetting of previously learned tasks. In contrast, RFT inherently preserves prior knowledge and achieves performance comparable to multi-task training. (2) RFT successfully protects and even enhances the model's general knowledge on standard benchmarks, while SFT degrades general model capabilities severely. Further analysis reveals that this stability is not primarily due to explicit mechanisms like KL penalty or chain-of-thought reasoning. We investigate RFT's learning dynamics and find that its selective update mechanism inherently prevents interference with established knowledge. Based on this insight, we propose a rollout-based instance filtering algorithm (RIF-RFT) that enhances the training efficiency of RFT by focusing on learnable samples. Our comprehensive study demonstrates the superiority of RFT as a robust paradigm for continual post-training.

cs.LG↗
arXiv2026

The Midas Touch for Metric Depth

Recent advances have markedly improved the cross-scene generalization of relative depth estimation, yet its practical applicability remains limited by the absence of metric scale, local inconsistencies, and low computational efficiency. To address these issues, we present \emph{\textbf{M}idas \textbf{T}ouch for \textbf{D}epth} (MTD), a mathematically interpretable approach that converts relative depth into metric depth using only extremely sparse 3D data. To eliminate local scale inconsistencies, it applies a segment-wise recovery strategy via sparse graph optimization, followed by a pixel-wise refinement strategy using a discontinuity-aware geodesic cost. MTD exhibits strong generalization and achieves substantial accuracy improvements over previous depth completion and depth estimation methods. Moreover, its lightweight, plug-and-play design facilitates deployment and integration on diverse downstream 3D tasks. Project page is available at https://mias.group/MTD.

cs.CV↗
arXiv2026

CL-VISTA: Benchmarking Continual Learning in Video Large Language Models

Video Large Language Models (Video-LLMs) require continual learning to adapt to non-stationary real-world data. However, existing benchmarks fall short of evaluating modern foundation models: many still rely on models without large-scale pre-training, and prevailing benchmarks typically partition a single dataset into sub-tasks, resulting in high task redundancy and negligible forgetting on pre-trained Video-LLMs. To address these limitations, we propose CL-VISTA, a benchmark tailored for continual video understanding of Video-LLMs. By curating 8 diverse tasks spanning perception, understanding, and reasoning, CL-VISTA induces substantial distribution shifts that effectively expose catastrophic forgetting. To systematically assess CL methods, we establish a comprehensive evaluation framework comprising 6 distinct protocols across 3 critical dimensions: performance, computational efficiency, and memory footprint. Notably, the performance dimension incorporates a general video understanding assessment to assess whether CL methods genuinely enhance foundational intelligence or merely induce task-specific overfitting. Extensive benchmarking of 10 mainstream CL methods reveals a fundamental trade-off: no single approach achieves universal superiority across all dimensions. Methods that successfully mitigate catastrophic forgetting tend to compromise generalization or incur prohibitive computational and memory overheads. We hope CL-VISTA provides critical insights for advancing continual learning in multimodal foundation models.

cs.CV↗
arXiv2026

Automating Skill Acquisition through Large-Scale Mining of Open-Source Agentic Repositories: A Framework for Multi-Agent Procedural Knowledge Extraction

The transition from monolithic large language models (LLMs) to modular, skill-equipped agents represents a fundamental architectural shift in artificial intelligence deployment. While general-purpose models demonstrate remarkable breadth in declarative knowledge, their utility in autonomous workflows is frequently constrained by insufficient specialized procedural expertise. This report investigates a systematic framework for automated acquisition of high-quality agent skills through mining of open-source repositories on platforms such as GitHub. We focus on the extraction of visualization and educational capabilities from state-of-the-art systems including TheoremExplainAgent and Code2Video, both utilizing the Manim mathematical animation engine. The framework encompasses repository structural analysis, semantic skill identification through dense retrieval, and translation to the standardized SKILL.md format. We demonstrate that systematic extraction from agentic repositories, combined with rigorous security governance and multi-dimensional evaluation metrics, enables scalable acquisition of procedural knowledge that augments LLM capabilities without requiring model retraining. Our analysis reveals that agent-generated educational content can achieve 40\% gains in knowledge transfer efficiency while maintaining pedagogical quality comparable to human-crafted tutorials.

cs.AI↗
arXiv2026

Scaling Laws for Educational AI Agents

While scaling laws for Large Language Models (LLMs) have been extensively studied along dimensions of model parameters, training data, and compute, the scaling behavior of LLM-based educational agents remains unexplored. We propose that educational agent capability scales not merely with the underlying model size, but through structured dimensions that we collectively term the Agent Scaling Law: role definition clarity, skill depth, tool completeness, runtime capability, and educator expertise injection. Central to this framework is AgentProfile, a structured JSON-based specification that serves as the mechanism enabling systematic capability growth of educational agents. We present EduClaw, a profile-driven multi-agent platform that operationalizes this scaling law, demonstrating its effectiveness through the construction and deployment of 330+ educational agent profiles encompassing 1,100+ skill modules across K-12 subjects. Our empirical observations suggest that educational agent performance scales predictably with profile structural richness. We identify two complementary scaling axes -- Tool Scaling and Skill Scaling -- as future directions, arguing that the path to more capable educational AI lies not solely in larger models, but in stronger structured capability systems.

cs.AI↗
arXiv2026

Reward Prediction with Factorized World States

Agents must infer action outcomes and select actions that maximize a reward signal indicating how close the goal is to being reached. Supervised learning of reward models could introduce biases inherent to training data, limiting generalization to novel goals and environments. In this paper, we investigate whether well-defined world state representations alone can enable accurate reward prediction across domains. To address this, we introduce StateFactory, a factorized representation method that transforms unstructured observations into a hierarchical object-attribute structure using language models. This structured representation allows rewards to be estimated naturally as the semantic similarity between the current state and the goal state under hierarchical constraint. Overall, the compact representation structure induced by StateFactory enables strong reward generalization capabilities. We evaluate on RewardPrediction, a new benchmark dataset spanning five diverse domains and comprising 2,454 unique action-observation trajectories with step-wise ground-truth rewards. Our method shows promising zero-shot results against both VLWM-critic and LLM-as-a-Judge reward models, achieving 60% and 8% lower EPIC distance, respectively. Furthermore, this superior reward quality successfully translates into improved agent planning performance, yielding success rate gains of +21.64% on AlfWorld and +12.40% on ScienceWorld over reactive system-1 policies and enhancing system-2 agent planning. Project Page: https://statefactory.github.io

cs.CL↗
arXiv2026

Three phases of odd robotic active matter

Nonreciprocal interactions in active matter are known to generate exotic mechanical behaviors such as odd elasticity and odd viscosity. However, these phenomena have largely been studied in isolation, raising a fundamental question: Is there a single system that embodies these distinct regimes of odd matter and can transition between phases, establishing a unified phase diagram for nonreciprocal active matter? To address this, we introduce a tunable robotic active matter platform, the Magnetomechanically Augmented Spinning roBotic (MASBot) collective, in which particle-level control of chirality, activity, and pairwise interactions enables access to distinct phases of odd matter. By continuously increasing repulsive forces relative to attractive and transverse forces, we experimentally map a transition from an odd elastic crystal to an odd viscous liquid, and then to a chiral active gas. We find that this latter phase forms a non-space-filling, nonreciprocal active gas stabilized by long-range hydrodynamic attractive forces, whose statistical signatures are consistent with those of a two-dimensional self-gravitating point vortex gas. Within these phases, adjusting spinning frequency and introducing spatially patterned activity allows us to fine-tune odd mechanical responses and tailor power spectra. Further polar and rotational symmetry breaking at the particle scale leads to novel emergent states such as phase separation and collective translation. Together, our system provides a fundamental experimental testbed for nonequilibrium physics and establishes a blueprint for treating robotic swarms as programmable states of matter, enabling functions that range from resilient structures to adaptive swarm reconfiguration.

cond-mat.soft↗
arXiv2026

Dynamic Instabilities and Pattern Formation in Chemotactic Active Matter

Collectives of actively-moving particles can spontaneously segregate into dilute and dense phases through a process known as motility-induced phase separation (MIPS). This captivating phenomenon is well-studied for randomly-moving particles with no directional bias. However, many active systems perform collective chemotaxis -- directed motion along a chemical gradient collectively generated by the particles themselves through consumption or production. Here, we use linear stability analysis, amplitude equations, and numerical simulations to study how MIPS is influenced by collective chemotaxis. We find that chemotaxis can either arrest or entirely suppress MIPS, or give rise to novel dynamic instabilities such as traveling waves and spirals. We predict the stability region of the stationary and oscillatory patterns and identify four types of bifurcation that can arise: pitchfork, saddle-node, infinite period, and supercritical Hopf. We also derive analytical expressions for the amplitude of the pattern and traveling wave velocity, yielding excellent quantitative agreement with simulations. Furthermore, we generalize our model to study particles that either consume or produce chemoattractant or chemorepellent, as well as mixtures of particles with different chemotactic behaviors. By establishing quantitative principles describing the competition between MIPS and chemotaxis, our study helps deepen understanding of the rich physics underlying chemically-responsive active matter systems.

cond-mat.soft↗
arXiv2026

Two-Stream Interactive Joint Learning of Scene Parsing and Geometric Vision Tasks

Inspired by the human visual system, which operates on two parallel yet interactive streams for contextual and spatial understanding, this article presents Two Interactive Streams (TwInS), a novel bio-inspired joint learning framework capable of simultaneously performing scene parsing and geometric vision tasks. TwInS adopts a unified, general-purpose architecture in which multi-level contextual features from the scene parsing stream are infused into the geometric vision stream to guide its iterative refinement. In the reverse direction, decoded geometric features are projected into the contextual feature space for selective heterogeneous feature fusion via a novel cross-task adapter, which leverages rich cross-view geometric cues to enhance scene parsing. To eliminate the dependence on costly human-annotated correspondence ground truth, TwInS is further equipped with a tailored semi-supervised training strategy, which unleashes the potential of large-scale multi-view data and enables continuous self-evolution without requiring ground-truth correspondences. Extensive experiments conducted on three public datasets validate the effectiveness of TwInS's core components and demonstrate its superior performance over existing state-of-the-art approaches. The source code will be made publicly available upon publication.

cs.CV↗
arXiv2026

Practical Continual Forgetting for Pre-trained Vision Models

For privacy and security concerns, the need to erase unwanted information from pre-trained vision models is becoming evident nowadays. In real-world scenarios, erasure requests originate at any time from both users and model owners, and these requests usually form a sequence. Therefore, under such a setting, selective information is expected to be continuously removed from a pre-trained model while maintaining the rest. We define this problem as continual forgetting and identify three key challenges. (i) For unwanted knowledge, efficient and effective deleting is crucial. (ii) For remaining knowledge, the impact brought by the forgetting procedure should be minimal. (iii) In real-world scenarios, the training samples may be scarce or partially missing during the process of forgetting. To address them, we first propose Group Sparse LoRA (GS-LoRA). Specifically, towards (i), we introduce Low-Rank Adaptation (LoRA) modules to fine-tune the Feed-Forward Network (FFN) layers in Transformer blocks for each forgetting task independently, and towards (ii), a simple group sparse regularization is adopted, enabling automatic selection of specific LoRA groups and zeroing out the others. To further extend GS-LoRA to more practical scenarios, we incorporate prototype information as additional supervision and introduce a more practical approach, GS-LoRA++. For each forgotten class, we move the logits away from its original prototype. For the remaining classes, we pull the logits closer to their respective prototypes. We conduct extensive experiments on face recognition, object detection, and image classification and demonstrate that our method manages to forget specific classes with minimal impact on other classes. Codes have been released on https://github.com/bjzhb666/GS-LoRA.

cs.CV↗
arXiv2025

MCITlib: Multimodal Continual Instruction Tuning Library and Benchmark

Continual learning enables AI systems to acquire new knowledge while retaining previously learned information. While traditional unimodal methods have made progress, the rise of Multimodal Large Language Models (MLLMs) brings new challenges in Multimodal Continual Learning (MCL), where models are expected to address both catastrophic forgetting and cross-modal coordination. To advance research in this area, we present MCITlib, a comprehensive library for Multimodal Continual Instruction Tuning. MCITlib currently implements 8 representative algorithms and conducts evaluations on 3 benchmarks under 2 backbone models. The library will be continuously updated to support future developments in MCL. The codebase is released at https://github.com/Ghy0501/MCITlib.

cs.CV↗