arXiv Science⌕ Search

arXiv subjects

Hui Zhang

Publications and source records attributed to Hui Zhang.

At least 55 records · Page 3Linked to original sources
arXiv2026

Medical Image Alignment Assessment as a Test of Generalist Visual Reasoning in Frontier Multimodal Models

Frontier multimodal large language models (MLLMs) are increasingly positioned as general purpose visual reasoners as part of the quest for artificial general intelligence. A key test of this generality is whether they can perform novel visual judgments that humans can make reliably from visual evidence and task instructions, without task-specific parameter optimisation. We investigate this question through the task of medical image alignment assessment, where the goal is to establish whether there is anatomical correspondence between two images. Human visual assessment of image alignment is still the gold standard and most common approach; however, it requires trained operators and is impractical to scale for large datasets. We evaluate recent generations of MLLMs on two exemplar medical image alignment tasks, varying both prompting strategies and image-presentation methods. We compare against a locally fine-tuned MLLM and a task-specific CNN to examine the trade-off between frontier general purpose models and smaller models that require specific task optimisation but can be used locally. We show that are reaching an inflection point, where frontier MLLMs can now perform effective visual assessment of medical image alignment. Models released only a few months ago generalise poorly and, in some settings, perform barely above chance, whereas GPT-6 achieves over 85% across almost all scenarios tested. Fine-tuned local models can match or exceed frontier-model performance on the tasks on which they are trained, but transfer substantially less effectively to unseen settings. These findings identify medical image alignment as a useful test bed for generalist visual reasoning and suggest that frontier multimodal models are beginning to acquire capabilities that could support a common quality-control mechanism across heterogeneous medical-imaging pipelines.

cs.CV↗
arXiv2026

AeRSoM: An Aerial Rigid-Soft Integrated Manipulator for Contact-Rich Manipulation

Contact-rich aerial manipulation remains fundamentally challenging because interaction forces are directly transmitted to the aerial platform, often leading to instability and degraded task performance. While compliant manipulators can mitigate these effects, existing aerial manipulation systems typically struggle to reconcile interaction compliance with manipulation precision. To this end, this article presents an aerial rigid-soft integrated manipulator (AeRSoM) robot that realizes embodied compliance for aerial manipulation. The proposed system integrates a fully actuated aerial platform, a rigid-soft manipulator, and variable-stiffness regulation to simultaneously achieve stable flight, compliant interaction, and precise manipulation. By distributing compliance throughout the manipulation system, the proposed design leverages distributed embodied compliance to passively absorb contact disturbances while preserving sufficient stiffness for task execution. To fully exploit the mechanical design, a composite control framework is developed for precise end-effector trajectory tracking in the presence of uncertainties and external disturbances. Extensive real-world experiments are conducted in representative contact-rich aerial manipulation tasks, including dynamic transmission-line grasping, physical interaction with a wind turbine blade, peg-in-hole, and screwing operations. The results demonstrate that the proposed rigid-soft integration significantly improves interaction robustness and task adaptability while maintaining manipulation accuracy, highlighting that embodied compliance provides a promising design paradigm for enhancing the safety, robustness, and versatility of aerial manipulation.

cs.RO↗
arXiv2026

High-order mass-, energy- and momentum-conserving methods for the nonlinear Schrödinger equation

This paper introduces a novel formulation and an associated space-time finite element method for simulating solutions to the nonlinear Schrödinger equation. A major advantage of the proposed algorithm is its intrinsic ability to preserve the conservation of mass, energy, and momentum at the discrete level. This is proved for the numerical solutions determined by the fully discrete implicit scheme. An effective iterative scheme is proposed for solving the nonlinear system based on an equivalent formulation which suggests using Newton's iteration for the solution and no iteration for the Lagrange multipliers in the nonlinear system. Extensive numerical examples are provided to demonstrate the high-order convergence and effectiveness of the proposed algorithm in conserving mass, energy, and momentum in the simulation of one-dimensional Ma-solitons and bi-solitons, as well as of two-dimensional solitons governed by the nonlinear Schrödinger equation. The numerical results show that the mass-, energy- and momentum-conserving method designed in this paper also significantly reduces the errors of the numerical solutions in long-time simulations compared with methods which do not conserve these quantities.

math.NA↗
arXiv2026

CommitFlow: Semantic Commitment Verification and Local Correction for Long-Horizon Robot Manipulation VLA Execution

Although vision-language-action (VLA) policies have advanced rapidly, long-horizon execution may still progress to the next task stage before the required physical effect has been established. We call this a mismatch between semantic commitments, physical conditions that a stage must establish or maintain, and the actual physical state. Because an action command alone cannot confirm such a condition, local deviations can propagate and cause task failure. To address this problem, we present CommitFlow, a closed-loop execution framework that combines commitment monitoring with local correction while keeping the base policy frozen. CommitFlow integrates three components. A Semantic Commitment Monitor (SCM) compares stage requirements against current state evidence and holds back dependent actions when a required condition is unmet or violated. BoundaryFlow then generates a local correction conditioned on the current state and base action, and Relation and Gain Calibration (RGC) selects the smallest correction strength that satisfies the relevant constraints. Across the ten common RoboTwin 2.0 benchmark tasks, CommitFlow achieves a mean success rate of 75.9 percent, improving on the base policy pi0.5 by 22.7 percent. Cross-policy experiments show consistent gains, pointing toward reliable long-horizon robot execution.

cs.RO↗
arXiv2026

High-capacity computing with self-rectification nonlinear optical neural processor

Artificial intelligence (AI) and neural networks have driven groundbreaking innovations across numerous disciplines. Optical computing offers the promise of unprecedented speed and energy efficiency in the post-Moore era; however, achieving efficient, practical nonlinear activation using all-optical approaches remains a challenge. Here, we present an optical nonlinear neural processing unit (ONNPU) that implements all-optical nonlinear activation through a self-rectification mechanism. The ONNPU architecture perfectly imitates the structure of digital neural networks, enabling seamless integration with the established deep learning ecosystem. We benchmark ONNPU across nine diverse tasks spanning decision, regression and generation, including accuracies of 98.07% on MNIST and 93.54% on Fashion-MNIST. When integrated into a 201-million-parameter Vision Transformer, ONNPU achieves 82.4% top-1 accuracy on full ImageNet classification (1,000 categories); when integrated into a 117-million-parameter decoder-only Transformer, ONNPU enables short-form story generation that outperforms GPT-2. By addressing more complex and diverse deep learning tasks, ONNPU paves the way toward practical optical machine intelligence, unleashing significant potential for high-performance optical computing.

physics.optics↗
arXiv2026

Domain-decomposed Evolutional Deep Neural Network with Random Features for Transient Pressure Diffusion with Discontinuous and High-Contrast Coefficients

Transient pressure diffusion in heterogeneous porous media becomes difficult to resolve efficiently when permeability is discontinuous and spans several orders of magnitude. We develop a domain-decomposed random-feature evolutional deep neural network (DD RF-EDNN) that separates spatial approximation from temporal evolution. Permeability-informed random features are compressed into an orthonormal pressure space, and a conservative finite-volume operator is projected onto this space so that only the reduced coordinates are advanced in time. This formulation retains the dissipative structure of the discrete flow problem without repeated neural-network optimization. The analysis quantifies dictionary and singular-value truncation errors, proves contractivity under an energy-consistent dissipativity correction, and derives a conditional grid-level estimate for homogeneous dynamics and time-independent data admitting a steady lifting. Experiments on low-permeability inclusions, high-conductivity channels and blocks, and three-dimensional grids demonstrate consistent accuracy across distinct coefficient structures. The method reaches a final-time relative $L^2$ error of $9.54\times10^{-4}$ in the principal benchmark and maintains errors on the order of $10^{-2}$ at a permeability contrast of $10^3$. Numerical diagnostics further show that the observed error balance and temporal convergence are consistent with the analysis. These results support DD RF-EDNN as a structure-preserving and interpretable reduced formulation for transient simulations on prescribed heterogeneous media.

math.NA↗
arXiv2026

Distributed Safe Cooperative Vector Field for Trajectory Curvature Constrained Multi-Robot Systems

Trajectory curvature constraints are inherent in practical multi-robot systems due to the limited turning capabilities of the robots. Without properly accounting for these constraints, robots may fail to accomplish assigned tasks, and their trajectories may diverge from the intended paths. This paper proposes a distributed safe cooperative vector field approach for multi-robot systems subject to trajectory curvature constraints. The proposed approach is composed of a cooperative vector field and a safety-oriented collision avoidance vector field, aiming to address the problems of cooperative motion and safe collision avoidance in multi-robot path-following tasks. A safety-oriented collision avoidance vector field with adaptively adjustable reactive boundary is developed to accommodate the kinematic curvature constraints of robots, thereby ensuring the physical feasibility of collision avoidance maneuvers. The proposed vector field requires only a single virtual variable from each neighboring robot to achieve cooperative motion and ensure both obstacle avoidance and inter-robot collision avoidance. The effectiveness of the proposed approach is validated through both simulations and real-world experiments on an actual multi-robot platform.

cs.RO↗
arXiv2026

ConGraspXL: Controllable Constraint-Conditioned Dexterous Grasping Motion Synthesis

Dexterous grasping is usually conducted for specific tasks, leading to heterogeneous constraints such as specific approach directions, desired contact regions, specified wrist trajectories, and functional hand poses. Our previous work, GraspXL, achieves scalable grasping motion synthesis for diverse objects and hand morphologies, while lacking controllability for synthesis under such various task-driven constraints. In this paper, we propose ConGraspXL, which extends GraspXL with controllable constraint-conditioned grasp motion synthesis that accommodates diverse task-driven constraints and their combinations. We introduce a hierarchical constraint formulation, enable flexible constraint composition with a masked residual interface, and improve control precision with dynamic hand centers and feed-forward wrist guidance. Without losing the strong generalization capabilities of GraspXL, ConGraspXL enables precise and flexible controllability for various individual constraints and their combinations, providing a plug-and-play low-level grasp controller for downstream applications such as whole-body grasp completion, functional grasping, and human-motion imitation.

cs.RO↗
arXiv2026

UniH$^3$: Unifying Hierarchical Homogeneity and Heterogeneity for All-in-One Medical Image Restoration

All-in-One medical image restoration (MedIR) aims to address diverse tasks across modalities and degradation types using a single universal model. Existing methods typically prioritize modeling inter-task heterogeneity (e.g., distinct data distributions and degradation types). However, they largely neglect the inherent homogeneity present in medical images, such as widely shared anatomical structures within and across modalities, which can be leveraged to ease model training and improve generalization. To this end, we propose UniH3, a novel framework that Unifies Hierarchical Homogeneity and Heterogeneity for all-in-one medical image restoration. Specifically, to comprehensively exploit homogeneity, we introduce a Hierarchical Homogeneity Memory (H2M) module that progressively distills intra- and inter-task homogeneity priors from high-quality images during training, and adaptively retrieves the most relevant priors tailored to the input for guided restoration. These retrieved priors are then injected into the restoration pipeline via an efficient Homogeneity-Guided Attention (HGA) mechanism. Furthermore, to comprehensively address heterogeneity, we design a Hierarchical Heterogeneity Balancer (H2B) that mitigates both inter- and intra-task conflicts during optimization, facilitating balanced and effective multi-task learning. Extensive experiments on two large-scale benchmarks, MedIR-2D-500K and MedIR-3D-3K, demonstrate that UniH3 achieves state-of-the-art performance on both all-in-one and single-task medical image restoration. We hope this work establishes a strong benchmark and advances the development of general-purpose medical image restoration models. Code is available at https://github.com/Yaziwel/UniH3.

cs.CV↗
arXiv2026

Assembling Two Parts in One Hand

A hallmark of human dexterity is the cooperative use of fingers, where different fingers take on distinct yet coordinated roles to accomplish fine manipu- lation, such as capping a pen with the hand that holds it. We study this finger-level coordination through in-hand assembly: mating two rigid objects within a single dexterous hand, with no second arm and no fixture. We present a reinforcement learning formulation to solve this problem in a unified framework, which is driven by a goal relative pose between the two parts. Finger coordination is shaped by a function-based auxiliary reward and regularized toward a single human reference pose, while domain randomization and a fusion of historical proprioception and object observation confer robustness to occlusion-induced estimation noise. The same recipe solves three different assembly tasks (Bottle, Syringe, and Marker). Trained purely in simulation, the policies transfer zero-shot to hardware with a single camera, demonstrating robustness to state-estimation errors caused by oc- clusion. Our experiments also reveal that in-hand assembly places demands on hand morphology and can serve as a benchmark for modern robotic hand systems. Videos and code are available at https://ltbgbird.github.io/in-hand-assembly-page/.

cs.RO↗
arXiv2026

Positive biorthogonal curvature on $S^2\times T^2$

We construct explicit Riemannian metrics with positive biorthogonal curvature on $S^2\times T^2$, answering a question of Bettiol. In particular, positive biorthogonal curvature on a closed four-manifold does not force its fundamental group to be virtually free.

math.DG↗
arXiv2026

Hi-Singers: A Comprehensive High-Quality Dataset for Expressive Audio-Driven Singing Head Synthesis

State-of-the-art models for audio-driven digital human generation have achieved photo-realistic results in talking-head synthesis. However, extending these models to singing-head synthesis remains challenging due to a significant Domain Gap: singing requires more exaggerated expressions, vivid jaw openings, and precise rhythmic synchronization. Current models, primarily trained on speech datasets, often struggle with "rhythmic drift" and constrained dynamics. To address this, we introduce Hi-Singers, the first large-scale, high-quality, in-the-wild video dataset specifically tailored for singing head synthesis. Hi-Singers undergoes a rigorous automated and manual filtering pipeline, ensuring strict thematic adherence, high-resolution rendering, and stable motion, resulting in 29,608 video segments totaling approximately 170 hours. We further establish a dedicated evaluation benchmark balanced across linguistic and musical styles. Extensive experiments across diverse architectures, including 3D-coefficient and diffusion-based models, demonstrate that Hi-Singers consistently and significantly improves performance across all dimensions. Specifically, it enables models to achieve superior visual realism, enhanced lip-sync consistency, and more precise rhythmic dynamics, effectively bridging the domain gap and setting a new performance standard for the singing synthesis task. The dataset is available at https://huggingface.co/datasets/CharlesZhang-USTC/Hi-Singers

cs.MM↗
arXiv2026

Agentic Visual Generation: From Generative Models to Agentic Control

Visual generation is evolving from generative models used through a single invocation into agentic control processes that can plan, select tools, inspect intermediate synthesized outputs, revise failures, and reuse prior experience. In most existing systems, the controller is an LLM or VLM, while visual generation models serve as tools or executors. However, existing work lacks a consistent criterion for determining when a generation system becomes agentic. Planning depth, tool use, multi-role collaboration, and reinforcement learning are often treated as evidence of agenticity, even though none of them necessarily determines which generation decisions the controller can make. We organize the field according to what the controller can directly control in the generation process. At L1 Conditioning Control, the controller prepares the input to a predetermined generator but does not control which visual operation is executed. At L2 Execution Control, it selects and invokes actual generation, editing, rendering, or other content-modifying operations. At L3 Outcome-Adaptive Control, it observes an intermediate outcome and uses that observation to change a subsequent operation within the current task. At L4 Experience-Adaptive Control, it retains experience from completed tasks and uses that experience to change decisions on future tasks. L0 Fixed Support separately denotes generators, editors, evaluators, reward models, benchmarks, and fixed pipelines without a deployed controller that makes generation-level decisions. These levels describe a progressively broader decision-making scope rather than model size, system complexity, output quality, tool or role count, or training method. Applying this framework across image, video, editing, 3D, world, slide, and user-interface generation reveals how controller capabilities have evolved and how their mechanisms are distributed across levels.

cs.CV↗
arXiv2026

WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing

Image generation and editing models have advanced rapidly, yet remain unreliable when prompts require external world knowledge. Bounded and long-tail parametric knowledge prevents direct or reason-then-generate approaches from recovering the required facts and visual appearances. Existing agentic generation and editing methods mitigate this limitation with retrieval tools, yet remain constrained by insufficient visual verification, overloaded policy models, and weak integration of retrieved textual and visual evidence. To address these limitations, we present WeAgent-MMGenEdit, a full-stack recipe including a multimodal harness, a scalable data construction pipeline, a comprehensive benchmark, and post-training methods for the agent policy and image backend. We first introduce WeAgent-Harness, a multimodal runtime with persistent evidence management and dedicated verification and integration tools that organize retrieved multimodal evidence into a dense carrier. Upon this, we develop a scalable pipeline for prompt synthesis and agentic trajectory collection, yielding 23K supervised trajectories and 14.7K RL tasks with three-layer verifiable checklists. We further introduce WeBench-MMGenEdit, a bilingual benchmark covering both knowledge-intensive image generation and multi-image editing. Finally, a two-sided post-training recipe based on SFT and RL improves the agent policy and image backend. Together, WeAgent-MMGenEdit enables a 30B-total/3B-active policy to outperform similarly sized policy models and approach the performance of a 1T-parameter agent.

cs.CV↗
arXiv2026

WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agents

Multimodal search agents extend parametric knowledge with newly emerging and long-tail evidence from the open web. Yet many existing agentic search environments often expose retrieved evidence only as text and omit tool-returned images from subsequent context, reducing visually grounded trajectories to text-only reasoning. Long-horizon interaction also compounds tool-call, response-length, timeout, and budget failures, which can discard salvageable trajectories, waste rollout computation, and disturb policy updates. To address these issues, we introduce WeAgent-Harness, a multimodal agentic harness that supports native text-vision interaction and runtime recovery. Retrieved images receive persistent disk references, allowing the model to inspect, process, and cite them throughout the trajectory. Based on this harness, we develop WeAgent-MMSearch, an integrated system spanning data construction, agentic post-training, and multimodal rollout. For data construction, a strong MLLM uses WeAgent-Harness to discover, synthesize, and verify MMSearch-style tasks and collect expert trajectories. During post-training, our Failure-Aware GSPO (FA-GSPO) recovers salvageable abnormal rollouts and filters invalid ones to improve bounded multimodal planning and search. We also introduce VisTarget-Bench, a 150-task human-verified benchmark that pairs each question with a held-out target image, distinguishing image-retrieval failures from visual-perception failures. Evaluation on VisTarget-Bench and seven public benchmarks shows that agentic post-training improves the average score by 19.22 points, enabling our model to outperform similarly sized open-source models and rival models with roughly ten times its parameter count.

cs.AI↗
arXiv2026

Dimensionality-induced critical phase transition in stochastic Lotka-Volterra equation: From statistical averaging to systemic tipping point

By analyzing a stochastic Lotka-Volterra (LV) equation, we show that the underlying logic behind the diversity-stability debate in ecology can be framed as a dimensionality-induced critical phase transition, that is, a transition separating the regime dominated by statistical averaging and a systemic tipping point. This phase transition framework unifies the opposing ecological predictions in the diversity-stability debate and sets an intrinsic diversity ceiling for stochastic ecological communities.

q-bio.PE↗
arXiv2026

VersaDB: A High-Performance AI Storage Database for Unifying Mutimodal Datasets

The AI field has been rapidly developing, leading to the emergence of a large number of AI training datasets of various types. These datasets contain different modalities, including text, images, audio, etc., and may come in various data storage formats. With the advancement of AI hardware, AI computation units like GPUs, TPUs, and NPUs can greatly accelerate the training speed of AI models, which in turn increases the demand for faster data processing. When using existing AI processing frameworks to handle datasets with different modalities and storage formats, processing speeds may be suboptimal due to issues such as data layout and the way users handle the data. Therefore, using a unified database to store multiple data formats can better manage and optimize data access. In this paper, we introduce VersaDB, a database designed specifically for AI datasets with various modalities. We implemented a page-based storage system, separating structured and unstructured data. Additionally, we generated B+ tree-based index files to accelerate data access. VersaDB supports automatic sharding and maintains a hierarchical metadata management system, with corresponding metadata maintained at the page, shard, and global levels, forming the foundation for the efficient operation of the database. We also focused on ease of use by providing APIs for directly converting datasets into VersaDB, as well as APIs for converting popular AI data storage formats (e.g., CSV, TFRecord, .bin) into VersaDB.Our experiments show that using VersaDB can achieve up to 5.35x acceleration and maintain consistent performance across different parallelism levels.

cs.CV↗
arXiv2026

Sharp-Peak Functions for Exactly Penalizing Binary Integer Programming

Unconstrained binary integer programming (UBIP) is a challenging optimization problem due to the presence of binary variables. To address the challenge, we introduce a novel class of functions named sharp-peak functions (SPFs), which equivalently reformulate the binary constraints as equality constraints, giving rise to an SPF-constrained optimization. Rather than solving this constrained reformulation directly, we focus on its associated penalty model. The established exact penalty theory shows that the global minimizers of UBIP and the penalty model coincide when the penalty parameter exceeds a threshold, a constant independent of the solution set of UBIP. To analyze the penalty model, we introduce Karush-Kuhn-Tucker (KKT) points and a new type of stationarity, referred to as P-stationarity, and provide a comprehensive characterization of its optimality conditions. We then develop an efficient algorithm called Sha-Peak based on the inexact alternating direction method of multipliers. It converges toa P-stationary point at a linear rate or terminates at such a point within finitely many steps. These results are established under appropriate parameter choices and a single mild assumption, namely, the local Lipschitz continuity of the gradient over a bounded box. Finally, numerical experiments demonstrate its nice performance in comparison to several established solvers.

math.OC↗