arXiv subjects

Lei Zhu

Publications and source records attributed to Lei Zhu.

At least 91 records · Page 5Linked to original sources

arXiv2026

GSINA: Improving Subgraph Extraction for Graph Invariant Learning via Graph Sinkhorn Attention

Graph invariant learning (GIL) seeks invariant relations between graphs and labels under distribution shifts. Recent works try to extract an invariant subgraph to improve out-of-distribution (OOD) generalization, yet existing approaches either lack explicit control over compactness or rely on hard top-$k$ selection that shrinks the solution space and is only partially differentiable. In this paper, we provide an in-depth analysis of the drawbacks of some existing works and propose a few general principles for invariant subgraph extraction: 1) separability, as encouraged by our sparsity-driven mechanism, to filter out the irrelevant common features; 2) softness, for a broader solution space; and 3) differentiability, for a soundly end-to-end optimization pipeline. Specifically, building on optimal transport, we propose Graph Sinkhorn Attention (GSINA), a fully differentiable, cardinality-constrained attention mechanism that assigns sparse-yet-soft edge weights via Sinkhorn iterations and induces node attention. GSINA provides explicit controls for separability and softness, and uses a Gumbel reparameterization to stabilize training. It convergence behavior is also theoretically studied. Extensive empirical experimental results on both synthetic and real-world

cs.LG↗

arXiv2026

Toward Real-World High-Precision Image Matting and Segmentation

High-precision scene parsing tasks, including image matting and dichotomous segmentation, aim to accurately predict masks with extremely fine details (such as hair). Most existing methods focus on salient, single foreground objects. While interactive methods allow for target adjustment, their class-agnostic design restricts generalization across different categories. Furthermore, the scarcity of high-quality annotation has led to a reliance on inharmonious synthetic data, resulting in poor generalization to real-world scenarios. To this end, we propose a Foreground Consistent Learning model, dubbed as FCLM, to address the aforementioned issues. Specifically, we first introduce a Depth-Aware Distillation strategy where we transfer the depth-related knowledge for better foreground representation. Considering the data dilemma, we term the processing of synthetic data as domain adaptation problem where we propose a domain-invariant learning strategy to focus on foreground learning. To support interactive prediction, we contribute an Object-Oriented Decoder that can receive both visual and language prompts to predict the referring target. Experimental results show that our method quantitatively and qualitatively outperforms SOTA methods.

cs.CV↗

arXiv2026

VQ-Seg: Vector-Quantized Token Perturbation for Semi-Supervised Medical Image Segmentation

Consistency learning with feature perturbation is a widely used strategy in semi-supervised medical image segmentation. However, many existing perturbation methods rely on dropout, and thus require a careful manual tuning of the dropout rate, which is a sensitive hyperparameter and often difficult to optimize and may lead to suboptimal regularization. To overcome this limitation, we propose VQ-Seg, the first approach to employ vector quantization (VQ) to discretize the feature space and introduce a novel and controllable Quantized Perturbation Module (QPM) that replaces dropout. Our QPM perturbs discrete representations by shuffling the spatial locations of codebook indices, enabling effective and controllable regularization. To mitigate potential information loss caused by quantization, we design a dual-branch architecture where the post-quantization feature space is shared by both image reconstruction and segmentation tasks. Moreover, we introduce a Post-VQ Feature Adapter (PFA) to incorporate guidance from a foundation model (FM), supplementing the high-level semantic information lost during quantization. Furthermore, we collect a large-scale Lung Cancer (LC) dataset comprising 828 CT scans annotated for central-type lung carcinoma. Extensive experiments on the LC dataset and other public benchmarks demonstrate the effectiveness of our method, which outperforms state-of-the-art approaches. Code available at: https://github.com/script-Yang/VQ-Seg.

cs.CV↗

arXiv2026

Real-Time Lane Detection via Efficient Feature Alignment and Covariance Optimization for Low-Power Embedded Systems

Real-time lane detection in embedded systems encounters significant challenges due to subtle and sparse visual signals in RGB images, often constrained by limited computational resources and power consumption. Although deep learning models for lane detection categorized into segmentation-based, anchor-based, and curve-based methods there remains a scarcity of universally applicable optimization techniques tailored for low-power embedded environments. To overcome this, we propose an innovative Covariance Distribution Optimization (CDO) module specifically designed for efficient, real-time applications. The CDO module aligns lane feature distributions closely with ground-truth labels, significantly enhancing detection accuracy without increasing computational complexity. Evaluations were conducted on six diverse models across all three method categories, including two optimized for real-time applications and four state-of-the-art (SOTA) models, tested comprehensively on three major datasets: CULane, TuSimple, and LLAMAS. Experimental results demonstrate accuracy improvements ranging from 0.01% to 1.5%. The proposed CDO module is characterized by ease of integration into existing systems without structural modifications and utilizes existing model parameters to facilitate ongoing training, thus offering substantial benefits in performance, power efficiency, and operational flexibility in embedded systems.

cs.CV↗

arXiv2026

MANGO:Natural Multi-speaker 3D Talking Head Generation via 2D-Lifted Enhancement

Current audio-driven 3D head generation methods mainly focus on single-speaker scenarios, lacking natural, bidirectional listen-and-speak interaction. Achieving seamless conversational behavior, where speaking and listening states transition fluidly remains a key challenge. Existing 3D conversational avatar approaches rely on error-prone pseudo-3D labels that fail to capture fine-grained facial dynamics. To address these limitations, we introduce a novel two-stage framework MANGO, which leveraging pure image-level supervision by alternately training to mitigate the noise introduced by pseudo-3D labels, thereby achieving better alignment with real-world conversational behaviors. Specifically, in the first stage, a diffusion-based transformer with a dual-audio interaction module models natural 3D motion from multi-speaker audio. In the second stage, we use a fast 3D Gaussian Renderer to generate high-fidelity images and provide 2D-level photometric supervision for the 3D motions through alternate training. Additionally, we introduce MANGO-Dialog, a high-quality dataset with over 50 hours of aligned 2D-3D conversational data across 500+ identities. Extensive experiments demonstrate that our method achieves exceptional accuracy and realism in modeling two-person 3D dialogue motion, significantly advancing the fidelity and controllability of audio-driven talking heads.

cs.CV↗

arXiv2025

Two-photon light-sheet live imaging at kilohertz frame rate using birefringence-based pulse splitting

Multiphoton microscopy is widely used for live imaging. However, its acquisition speed remains limited by fluorophore emission rates and photodamage. To increase the pixel rate of a two-photon microscope beyond a few megahertz (MHz), multi-point parallelized schemes have been proposed. Two-photon (2P) light-sheet microscopy emerges as an effective approach for high-speed multiphoton imaging of live specimens, as it enables parallelized excitation while minimizing the required laser power. However, optimizing the signal-to-photodamage ratio in 2P light-sheet microscopy requires to precisely control the illumination parameters, including both wavelength and pulse frequency. Since conventional femtosecond laser sources generally do not allow independent modulation of these parameters, the development of low-cost, efficient and robust strategies to modulate the temporal excitation profile is essential to fully exploit the advantages of 2P light-sheet microscopy. Here, we introduce a compact pulse splitting scheme that meets these criteria. We used cascaded birefringent crystals to convert each excitation laser pulse into an adjustable sequence of collinear sub-pulses. We demonstrate its effectiveness in optimizing 2P light-sheet imaging of live zebrafish embryos. We analyze the impact of pulse splitting on photobleaching, nonlinear photodamage, and imaging performance. Additionally, we demonstrate high-speed 2P imaging of the beating heart and brain calcium dynamics using red fluorophores in live embryos. We achieve kilohertz imaging frame rate, reaching more than 150 MHz pixel rates with fluorescent signal levels above 10 $photons.pixel^{-1}$ using a laser mean power and a peak intensity in the range of 100 mW and 0.1 $TW.cm^{-2}$ at the sample, respectively. This adjustable pulse-splitting scheme allows full advantage to be taken of light-sheet illumination for fast in vivo 2P imaging.

physics.optics↗

arXiv2025

Provable Ordering and Continuity in Vision-Language Pretraining for Generalizable Embodied Agents

Pre-training vision-language representations on human action videos has emerged as a promising approach to reduce reliance on large-scale expert demonstrations for training embodied agents. However, prior methods often employ time contrastive learning based on goal-reaching heuristics, progressively aligning language instructions from the initial to the final frame. This overemphasis on future frames can result in erroneous vision-language associations, as actions may terminate early or include irrelevant moments in the end. To address this issue, we propose Action Temporal Coherence Learning (AcTOL) to learn ordered and continuous vision-language representations without rigid goal-based constraint. AcTOL treats a video as a continuous trajectory where it (1) contrasts semantic differences between frames to reflect their natural ordering, and (2) imposes a local Brownian bridge constraint to ensure smooth transitions across intermediate frames. Extensive imitation learning experiments on both simulated and real robots show that the pretrained features significantly enhance downstream manipulation tasks with high robustness to different linguistic styles of instructions, offering a viable pathway toward generalized embodied agents.

cs.RO↗

arXiv2025

Inter- and Intra-image Refinement for Few Shot Segmentation

Deep neural networks for semantic segmentation rely on large-scale annotated datasets, leading to an annotation bottleneck that motivates few shot semantic segmentation (FSS) which aims to generalize to novel classes with minimal labeled exemplars. Most existing FSS methods adopt a prototype-based paradigm, which generates query prior map by extracting masked-area features from support images and then makes predictions guided by the prior map. However, they suffer from two critical limitations induced by inter- and intra-image discrepancies: 1) The intra-class gap between support and query images, caused by single-prototype representation, results in scattered and noisy prior maps; 2) The inter-class interference from visually similar but semantically distinct regions leads to inconsistent support-query feature matching and erroneous predictions. To address these issues, we propose the Inter- and Intra-image Refinement (IIR) model. The model contains an inter-image class activation mapping based method that generates two prototypes for class-consistent region matching, including core discriminative features and local specific features, and yields an accurate and robust prior map. For intra-image refinement, a directional dropout mechanism is introduced to mask inconsistent support-query feature pairs in cross attention, thereby enhancing decoder performance. Extensive experiments demonstrate that IIR achieves state-of-the-art performance on 9 benchmarks, covering standard FSS, part FSS, and cross-domain FSS. Our source code is available at \href{https://github.com/forypipi/IIR}{https://github.com/forypipi/IIR}.

cs.CV↗

arXiv2025

Relational Anatomical Supervision for Accurate 3D Multi-Chamber Cardiac Mesh Reconstruction

Accurate reconstruction of multi-chamber cardiac anatomy from medical images is a cornerstone for patient-specific modeling, physiological simulation, and interventional planning. However, current reconstruction pipelines fundamentally rely on surface-wise geometric supervision and model each chamber in isolation, resulting in anatomically implausible inter-chamber violations despite apparently favorable overlap or distance metrics. In this work, we propose a relational anatomical supervision framework for multi-chamber cardiac mesh reconstruction by introducing a Mesh Interrelation Enhancement (MIE) loss. The proposed formulation explicitly encodes spatial relationships between cardiac structures into a differentiable occupancy-based objective, thereby transforming qualitative anatomical rules into quantitative geometric supervision. We further establish violation-aware evaluation metrics to directly quantify inter-chamber structural correctness, revealing systematic limitations of commonly used geometric measures such as Dice and Chamfer distance. Extensive experiments on multi-center CT data, densely sampled MR data, and two independent external cohorts, including a highly heterogeneous congenital heart disease population, demonstrate that the proposed method consistently suppresses clinically critical boundary violations by up to 83\%, while maintaining competitive volumetric accuracy and achieving superior surface fidelity. Notably, the proposed relational supervision generalizes robustly across imaging modalities, centers, and pathological conditions, even under severe anatomical deformation. These results demonstrate that distance-based supervision alone is insufficient to guarantee anatomically faithful reconstruction, and that explicit enforcement of multi-structure anatomical relations provides a principled and robust pathway toward reliable patient-specific cardiac modeling.

cs.CV↗

arXiv2025

More than Segmentation: Benchmarking SAM 3 for Segmentation, 3D Perception, and Reconstruction in Robotic Surgery

The recent SAM 3 and SAM 3D have introduced significant advancements over the predecessor, SAM 2, particularly with the integration of language-based segmentation and enhanced 3D perception capabilities. SAM 3 supports zero-shot segmentation across a wide range of prompts, including point, bounding box, and language-based prompts, allowing for more flexible and intuitive interactions with the model. In this empirical evaluation, we assess the performance of SAM 3 in robot-assisted surgery, benchmarking its zero-shot segmentation with point and bounding box prompts and exploring its effectiveness in dynamic video tracking, alongside its newly introduced language prompt segmentation. While language prompts show potential, their performance in the surgical domain is currently suboptimal, highlighting the need for further domain-specific training. Additionally, we investigate SAM 3D's depth reconstruction abilities, demonstrating its capacity to process surgical scene data and reconstruct 3D anatomical structures from 2D images. Through comprehensive testing on the MICCAI EndoVis 2017 and EndoVis 2018 benchmarks, SAM 3 shows clear improvements over SAM and SAM 2 in both image and video segmentation under spatial prompts, while the zero-shot evaluations of SAM 3D on SCARED, StereoMIS, and EndoNeRF indicate strong monocular depth estimation and realistic 3D instrument reconstruction, yet also reveal remaining limitations in complex, highly dynamic surgical scenes.

cs.CV↗

arXiv2025

ALMA-QUARKS: Few-Thousand-Year Hatching out of "Egg": The Supersonic Breakout of a Hypercompact H II Region from Its Parental Hot Core

The kinematic evolution of hypercompact H II (HC H II) regions around young high-mass stars remains poorly understood due to complex interactions with parental environs. We present ALMA QUARKS/ATOMS 1.3 mm/3 mm observations (the highest resolution $\sim0.01$ pc) of a deeply embedded HC H II region (diameter $\sim0.015$ pc, electron density $\sim2\times10^{5}$ cm$^{-3}$) exhibiting a striking $\gtrsim20$ km s$^{-1}$ global redshift seen in optically thin H30$α$/H40$α$ recombination lines relative to its parental hot molecular core within a hub-filament system. The 1.3 mm continuum data reveal a distinct 0.1-pc arc and a perpendicular 0.04-pc tail. We propose that this morphology arises from a dynamic champagne flow: the slow expansion of HC H II region into a pre-existing filament forms the arc and associated low-velocity (few km s$^{-1}$) SiO shocks. Meanwhile, in the opposite direction ionized gas escapes along a steep density gradient traced by the tail and high-velocity (20 km s$^{-1}$) SiO emission. We reject the bow shock scenario in which ionized gas co-moves with a runaway high-mass star because shocked gas in the arc aligns with the hub velocity, contradicting the bow shock prediction. Non-LTE radiative transfer modeling further rules out infall of ionized gas as the velocity shift origin. We conclude that this exceptional HC H II region is undergoing a few-thousand-year transition phase of "hatching out of the egg": the ionized gas of HC H II region has just broken out of its parental hot core and now is flowing outward supersonically. This work highlights how anisotropic density distributions induce supersonically anisotropic ionized flows that govern HC H II region evolution.

astro-ph.GA↗

arXiv2025

AdaTok: Adaptive Token Compression with Object-Aware Representations for Efficient Multimodal LLMs

Multimodal Large Language Models (MLLMs) have demonstrated substantial value in unified text-image understanding and reasoning, primarily by converting images into sequences of patch-level tokens that align with their architectural paradigm. However, patch-level tokenization leads to a quadratic growth in image tokens, burdening MLLMs' understanding and reasoning with enormous computation and memory. Additionally, the traditional patch-wise scanning tokenization workflow misaligns with the human vision cognition system, further leading to hallucination and computational redundancy. To address this issue, we propose an object-level token merging strategy for Adaptive Token compression, revealing the consistency with human vision system. The experiments are conducted on multiple comprehensive benchmarks, which show that our approach averagely, utilizes only 10% tokens while achieving almost 96% of the vanilla model's performance. More extensive experimental results in comparison with relevant works demonstrate the superiority of our method in balancing compression ratio and performance. Our code will be available.

cs.CV↗

arXiv2025

UltraFlux: Data-Model Co-Design for High-quality Native 4K Text-to-Image Generation across Diverse Aspect Ratios

Diffusion transformers have recently delivered strong text-to-image generation around 1K resolution, but we show that extending them to native 4K across diverse aspect ratios exposes a tightly coupled failure mode spanning positional encoding, VAE compression, and optimization. Tackling any of these factors in isolation leaves substantial quality on the table. We therefore take a data-model co-design view and introduce UltraFlux, a Flux-based DiT trained natively at 4K on MultiAspect-4K-1M, a 1M-image 4K corpus with controlled multi-AR coverage, bilingual captions, and rich VLM/IQA metadata for resolution- and AR-aware sampling. On the model side, UltraFlux couples (i) Resonance 2D RoPE with YaRN for training-window-, frequency-, and AR-aware positional encoding at 4K; (ii) a simple, non-adversarial VAE post-training scheme that improves 4K reconstruction fidelity; (iii) an SNR-Aware Huber Wavelet objective that rebalances gradients across timesteps and frequency bands; and (iv) a Stage-wise Aesthetic Curriculum Learning strategy that concentrates high-aesthetic supervision on high-noise steps governed by the model prior. Together, these components yield a stable, detail-preserving 4K DiT that generalizes across wide, square, and tall ARs. On the Aesthetic-Eval at 4096 benchmark and multi-AR 4K settings, UltraFlux consistently outperforms strong open-source baselines across fidelity, aesthetic, and alignment metrics, and-with a LLM prompt refiner-matches or surpasses the proprietary Seedream 4.0.

cs.CV↗

arXiv2025

The ALMA-QUARKS survey: Evidence of a candidate high-mass prestellar core aside a bright-rimmed cloud IRAS 18290-0924

Although frequently reported in observations, the definitive confirmation of high-mass prestellar cores has remained elusive, presenting a persistent challenge in star formation studies. Using two-band observational data from the 3mm ATOMS and 1.3mm QUARKS surveys, we report a high-mass prestellar core candidate, C2, located on the side of the bright-rimmed cloud IRAS 18290-0924. The C2 core identified from the 3mm continuum data of the ATOMS survey ($\sim$2 arcsecond, $\rm\sim 10000~au$ at 5.3 kpc) has a mass ranging from 27-68 $M_{\odot}$ for temperatures 10-22K within a radius of $\sim$2800 au. The highest-resolution ($\sim$0.3 arcsecond, $\rm\sim 1500 au$) observations of this source presented to date from the QUARKS survey reveal no evidence of further fragmentation. Further analysis of a total $\sim$10 GHz band width of molecular line survey does not find star-formation activity (e.g., outflows, ionized gas) associated with the core, with a few molecular lines of cold gas detected only. Additionally, virial analysis indicates the C2 core is gravitationally bound ($α_{\rm vir} \sim0.1-0.3$) and thus could be undergoing collapse toward star formation. These results strongly establish a candidate for a high-mass prestellar core, contributing to the very limited number of such sources known to date.

astro-ph.GA↗

arXiv2025

A Step Toward World Models: A Survey on Robotic Manipulation

Autonomous agents are increasingly expected to operate in complex, dynamic, and uncertain environments, performing tasks such as manipulation, navigation, and decision-making. Achieving these capabilities requires agents to understand the underlying mechanisms and dynamics of the world, moving beyond reactive control or simple replication of observed states. This motivates the development of world models as internal representations that encode environmental states, capture dynamics, and support prediction, planning, and reasoning. Despite growing interest, the definition, scope, architectures, and essential capabilities of world models remain ambiguous. In this survey, we go beyond prescribing a fixed definition and limiting our scope to methods explicitly labeled as world models. Instead, we examine approaches that exhibit the core capabilities of world models through a review of methods in robotic manipulation. We analyze their roles across perception, prediction, and control, identify key challenges and solutions, and distill the core components, capabilities, and functions that a fully realized world model should possess. Building on this analysis, we aim to motivate further development toward generalizable and practical world models for robotics.

cs.RO↗

arXiv2025

K-Stain: Keypoint-Driven Correspondence for H&E-to-IHC Virtual Staining

Virtual staining offers a promising method for converting Hematoxylin and Eosin (H&E) images into Immunohistochemical (IHC) images, eliminating the need for costly chemical processes. However, existing methods often struggle to utilize spatial information effectively due to misalignment in tissue slices. To overcome this challenge, we leverage keypoints as robust indicators of spatial correspondence, enabling more precise alignment and integration of structural details in synthesized IHC images. We introduce K-Stain, a novel framework that employs keypoint-based spatial and semantic relationships to enhance synthesized IHC image fidelity. K-Stain comprises three main components: (1) a Hierarchical Spatial Keypoint Detector (HSKD) for identifying keypoints in stain images, (2) a Keypoint-aware Enhancement Generator (KEG) that integrates these keypoints during image generation, and (3) a Keypoint Guided Discriminator (KGD) that improves the discriminator's sensitivity to spatial details. Our approach leverages contextual information from adjacent slices, resulting in more accurate and visually consistent IHC images. Extensive experiments show that K-Stain outperforms state-of-the-art methods in quantitative metrics and visual quality.

cs.CV↗

arXiv2025

The ALMA-QUARKS survey: Hot Molecular Cores are a long-standing phenomenon in the evolution of massive protostars

We present an analysis of the QUARKS survey sample, focusing on protoclusters where Hot Molecular Cores (HMCs, traced by CH3CN(12--11)) and UC HII regions (traced by H30α/H40α) coexist. Using the high-resolution, high-sensitivity 1.3 mm data from the QUARKS survey, we identify 125 Hot Molecular Fragments (HMFs), which represent the substructures of HMCs at higher resolution. From line integrated intensity maps of CH3CN(12--11) and H30α, we resolve the spatial distribution of HMFs and UC HII regions. By combining with observations of CO outflows and 1.3 mm continuum, we classify HMFs into four types: HMFs associated with jet-like outflow, with wide-angle outflow, with non-detectable outflow, and shell-like HMFs near UC HII regions. This diversity possibly indicates that the hot core could be polymorphic and long-standing phenomenon in the evolution of massive protostars. The separation between HMFs and H30α/H40αemission suggests that sequential high-mass star formation within young protoclusters is not likely related to feedback mechanisms.

astro-ph.GA↗

arXiv2025

BLM$_1$: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning

Multimodal large language models (MLLMs) have advanced vision-language reasoning and are increasingly deployed in embodied agents. However, significant limitations remain: MLLMs generalize poorly across digital-physical spaces and embodiments; vision-language-action models (VLAs) produce low-level actions yet lack robust high-level embodied reasoning; and most embodied large language models (ELLMs) are constrained to digital-space with poor generalization to the physical world. Thus, unified models that operate seamlessly across digital and physical spaces while generalizing across embodiments and tasks remain absent. We introduce the \textbf{Boundless Large Model (BLM$_1$)}, a multimodal spatial foundation model that preserves instruction following and reasoning, incorporates embodied knowledge, and supports robust cross-embodiment control. BLM$_1$ integrates three key capabilities -- \textit{cross-space transfer, cross-task learning, and cross-embodiment generalization} -- via a two-stage training paradigm. Stage I injects embodied knowledge into the MLLM through curated digital corpora while maintaining language competence. Stage II trains a policy module through an intent-bridging interface that extracts high-level semantics from the MLLM to guide control, without fine-tuning the MLLM backbone. This process is supported by a self-collected cross-embodiment demonstration suite spanning four robot embodiments and six progressively challenging tasks. Evaluations across digital and physical benchmarks show that a single BLM$_1$ instance outperforms four model families -- MLLMs, ELLMs, VLAs, and GMLMs -- achieving $\sim\!\textbf{6%}$ gains in digital tasks and $\sim\!\textbf{3%}$ in physical tasks.

cs.AI↗