arXiv subjects

Xiang Zhu

Publications and source records attributed to Xiang Zhu.

At least 19 recordsLinked to original sources

arXiv2026

Video Prediction Policy 2: Predict Better, Act Better

World action models (WAMs) have emerged as an important class of generalist robot policies, aiming to transfer video prediction priors to action learning. However, we find that existing WAMs frequently produce incorrect motion predictions in open-ended environment, leading to erroneous actions. We attribute this limitation to two factors: (1) base video models are not optimized for manipulation, and (2) naively incorporating action components into video models can substantially degrade their generalization capabilities. We introduce Video Prediction Policy 2 (VPP2), a WAM that enables strong zero-shot generalization in both video prediction and action generation. First, we curate a large-scale, diverse dataset of manipulation videos to continue pretraining the base video foundation model. We annotate video clips with detailed captions and perform \textit{event-level} video pretraining to promote generalization across open-ended manipulation tasks. Second, we post-train and distill the video model into a single-step visual planner with fixed prediction horizon. Finally, we introduce action module via a mixture-of-transformers (MoT) architecture to learn implicit inverse dynamics model. Experiments demonstrate three key results: (1) VPP2-14B outperforms Cosmos3-64B by 11.0\% points in video prediction instruction-following success rate on open-ended tasks; (2) VPP2 surpasses the strongest baseline by 18.5\% points in success rate on real-world zero-shot ALOHA manipulation tasks; and (3) following benchmark-specific post-training, VPP2 achieves the highest success rates among evaluated methods on the challenging LIBERO-Pro, LIBERO-OOD, and RoboDojo benchmarks.

cs.CV↗

arXiv2026

HARP-VLA: Human-Robot Aligned Representation Learning for Vision-Language-Action Model

Learning generalizable vision-language-action (VLA) models from large-scale human videos is promising but challenging due to cross-embodiment discrepancies in both visual observations and executable actions. While latent action models reduce the action execution gap by learning action abstractions, they still rely on visual features. Thus, misaligned human and robot visual representations can lead to inconsistencies in policy inputs and induce domain-dependent latent actions, hindering effective co-training with human videos.To address this, we propose HARP, a human-robot aligned representation learning framework for more effective VLA pretraining from human videos. Specifically, HARP uses paired human-robot demonstrations as cross-embodiment bridges and abundant unpaired human and robot videos as a scalable dynamics supervision data source. It trains a robot-adapted visual encoder and a latent action model with manipulation-centric auxiliary cues and a source-relative pair-discriminative alignment loss, which adapts robot representations toward human semantics while preserving pair-level discrimination. The learned aligned vision encoder and latent action model provide a unified vision and action representation for VLA-style policy learning, where human and robot videos provide vision-language-to-latent-action supervision and a lightweight robot action head grounds latent actions into executable commands. Experiments on feature visualization, simulation, and real-world manipulation show improved human-robot alignment and downstream policy performance, achieving 4.481 average length on CALVIN ABC$\rightarrow$D and a 7.1 percentage-point improvement in real-world success rate over the strongest evaluated baseline. Code, videos, and demos are available at https://puzhenyuan.github.io/HARP-VLA-website/.

cs.RO↗

arXiv2026

GALA: Geometry-Aware Latent Action Modeling for Vision-Language-Action Model Pretraining across Embodiments

Learning large-scale vision-language-action (VLA) models from multi-embodiment datasets remains challenging due to heterogeneous action spaces across end effectors. Although latent action models (LAMs) can learn embodiment-agnostic action representations from diverse video data, existing image-based LAMs often fail to capture fine-grained end-effector articulation, particularly finger-level geometric changes in human and dexterous robot hands. To address this limitation, we propose GALA, a Geometry-Aware Latent-Action modeling framework that augments image-based latent actions with 3D end-effector geometric motion. However, naively incorporating point clouds yields fine-grained action representations with limited shared semantics, hindering cross-embodiment pretraining. To address this issue, we introduce the Unified End-effector Motion Representation (UEMR), which preserves fine-grained motion information while improving the cross-embodiment generalizability of latent actions. Building upon UEMR, GALA combines visual latent actions that capture scene-level dynamics with geometric latent actions that capture shared fine-grained end-effector articulation, providing effective supervision for VLA pretraining from multi-embodiment data, including action-free ego-centric human videos. Experiments on fine-grained motion probing, cross-embodiment retrieval, and downstream VLA evaluation demonstrate GALA's effectiveness in modeling generalizable fine-grained motions across embodiments, achieving 68.3% RoboCasa-GR1 success rate and 75.5% real-world success rate. Code, appendix, and demos are available at https://puzhenyuan.github.io/GALA-website/.

cs.RO↗

arXiv2026

EVEREST:Endogenous Vision-Language Reinforcement Reasoning Exploration for Urban Socio-Semantic Segmentation

Urban socio-semantic segmentation leverages digital and satellite imagery to provide critical spatial semantic information for downstream applications such as urban resource allocation. Although existing methods achieve high segmentation accuracy, they still suffer from inaccurate delineation of target boundaries. The underlying issue is that current models primarily rely on passively aggregated global cross-modal cues, lacking active exploration of the environment. To address this limitation, we propose the EVEREST model, which adopts an egocentric exploration strategy that enables the model to actively investigate boundary cues and perform self-correction. In addition, we formulate discrete natural-language prompts as pseudocode to regularize the execution logic. Reinforcement learning is further employed to implement this irreducible process and elicit the model's structured reasoning capability. Our EVEREST achieves optimal performance on all metrics in the real world urban socio-semantic dataset, demonstrating the superiority of our model. Codes are available at https://github.com/TechCloud-x/EVEREST.

cs.MM↗

arXiv2026

TIAO: Token Importance-Aware Policy Optimization for Text Summarization

Text summarization requires models to condense content while preserving key qualities such as consistency and coherence. Large language models (LLMs) have shown strong performance on this task and can be further improved through reinforcement learning (RL). However, most existing methods apply reward signals directly to undifferentiated token sequences, overlooking the varying importance of individual tokens to word and sentence level quality in summarization. In this paper, we propose Token Importance-Aware Policy Optimization (TIAO), a novel reinforcement learning strategy that explicitly leverages token-importance awareness. Specifically, TIAO identifies core tokens based on token dependency and reweights a trajectory's advantage according to its overall dependencies. Experiments on the real world dataset show that our TIAO achieves highly competitive results, and that a 7B foundation model enhanced by TIAO performs comparably to GPT-4 and GPT-5-nano. Code is available at https://github.com/TechCloud-x/TIAO

cs.CL↗

arXiv2026

Multiparameter quantum estimation in a photon system induced by gravitational redshift

As photons propagate through curved spacetime, gravitational effects become unavoidable. In particular, gravitational redshift can induce significant distortion in photon wave packets, making it es?sential to investigate parameter estimation within this context. While previous research has focused on single-parameter estimation using the quantum Cramer-Rao bound, the multiparameter scenario remains largely unexplored. In this work, we investigate multiparameter quantum estimation for a photon system subject to gravitational redshift under both amplitude-damping and Ohmic-like dephasing channels. Our analysis reveals that the quantum Cramer-Rao bound fails to provide a tight error bound for the two-parameter estimation involving the initial phase and weight parameters inboth types of noisy channels. To overcome this limitation, we numerically compute two tighter error bounds, i.e., the Holevo Cramer-Rao bound and the Nagaoka bound, when utilizing a semidefinite program. We demonstrate that the Nagaoka bound yields the tightest error bound among all considered bounds, consistent with the general hierarchy of multiparameter quantum estimation. Furthermore, for the three-parameter estimation, including the initial weight parameter, the phase parameter, and the strength of gravitational redshift, we observe significantly enhanced estimation precision in the strong-coupling regime compared to the weak-coupling regime under the amplitude-damping channel. Similarly, in the Ohmic-like dephasing channel, the sub-Ohmic regime consistently affords higher precision than the Ohmic and super-Ohmic regimes.

quant-ph↗

arXiv2026

Learning Generalizable Robot Policy with Human Demonstration Video as a Prompt

Recent robot learning methods commonly rely on imitation learning from massive robotic dataset collected with teleoperation. When facing a new task, such methods generally require collecting a set of new teleoperation data and finetuning the policy. Furthermore, the teleoperation data collection pipeline is also tedious and expensive. Instead, human is able to efficiently learn new tasks by just watching others do. In this paper, we introduce a novel two-stage framework that utilizes human demonstrations to learn a generalizable robot policy. Such policy can directly take human demonstration video as a prompt and perform new tasks without any new teleoperation data and model finetuning at all. In the first stage, we train video generation model that captures a joint representation for both the human and robot demonstration video data using cross-prediction. In the second stage, we fuse the learned representation with a shared action space between human and robot using a novel prototypical contrastive loss. Empirical evaluations on real-world dexterous manipulation tasks show the effectiveness and generalization capabilities of our proposed method.

cs.RO↗

arXiv2026

Decentralized Frequency-Domain Conditions for D-Stability with Application to DC Microgrids

This paper proposes a decentralized method for regional pole placement, or $\mathcal{D}$-stability, in linearized networked systems. Existing LMI-based methods are hindered by confidentiality concerns regarding proprietary subsystem models and the absence of communication infrastructures. To overcome these barriers, we map the target region $\mathcal{D}$ of pole placement to an auxiliary left-half plane and introduce positive functions to handle the resulting complex-coefficient dynamics. We prove that $\mathcal{D}$-stability is guaranteed via local frequency-domain criteria without requiring shared subsystem models or inter-subsystem communication. This method is then tailored to DC microgrids, where a loop transformation is utilized to reallocate the burden of stability certification, deriving a broadcastable grid code for decentralized parameter synthesis. Numerical examples verify the efficacy of the proposed method.

eess.SY↗

arXiv2026

Decentralized Analysis Approach for Oscillation Damping in Grid-Forming and Grid-Following Heterogeneous Power Systems

This letter proposes a decentralized local gain condition (LGC) to guarantee oscillation damping in inverter-based resource (IBR)-dominated power systems. The LGC constrains the dynamic gain between each IBR and the network at its point of connection. By satisfying the LGC locally, the closed-loop poles are confined to a desired region, thereby yielding system-wide oscillation damping without requiring global information. Notably, the LGC is agnostic to different IBR dynamics, well-suited for systems with heterogeneous IBRs, and flexible to various damping requirements. Moreover, a low-complexity algorithm is proposed to parameterize LGC, providing scalable and damping-constrained parameter tuning guidance for IBRs.

eess.SY↗

arXiv2025

Diffusion Models Bridge Deep Learning and Physics in ENSO Forecasting

Accurate long-range forecasting of the El \Nino-Southern Oscillation (ENSO) is vital for global climate prediction and disaster risk management. Yet, limited understanding of ENSO's physical mechanisms constrains both numerical and deep learning approaches, which often struggle to balance predictive accuracy with physical interpretability. Here, we introduce a data driven model for ENSO prediction based on conditional diffusion model. By constructing a probabilistic mapping from historical to future states using higher-order Markov chain, our model explicitly quantifies intrinsic uncertainty. The approach achieves extending lead times of state-of-the-art methods, resolving early development signals of the spring predictability barrier, and faithfully reproducing the spatiotemporal evolution of historical extreme events. The most striking implication is that our analysis reveals that the reverse diffusion process inherently encodes the classical recharge-discharge mechanism, with its operational dynamics exhibiting remarkable consistency with the governing principles of the van der Pol oscillator equation. These findings establish diffusion models as a new paradigm for ENSO forecasting, offering not only superior probabilistic skill but also a physically grounded theoretical framework that bridges data-driven prediction with deterministic dynamical systems, thereby advancing the study of complex geophysical processes.

physics.geo-ph↗

arXiv2025

Rethinking Video Super-Resolution: Towards Diffusion-Based Methods without Motion Alignment

In this work, we rethink the approach to video super-resolution by introducing a method based on the Diffusion Posterior Sampling framework, combined with an unconditional video diffusion transformer operating in latent space. The video generation model, a diffusion transformer, functions as a space-time model. We argue that a powerful model, which learns the physics of the real world, can easily handle various kinds of motion patterns as prior knowledge, thus eliminating the need for explicit estimation of optical flows or motion parameters for pixel alignment. Furthermore, a single instance of the proposed video diffusion transformer model can adapt to different sampling conditions without re-training. Empirical results on synthetic and real-world datasets illustrate the feasibility of diffusion-based, alignment-free video super-resolution.

cs.CV↗

arXiv2025

UP-VLA: A Unified Understanding and Prediction Model for Embodied Agent

Recent advancements in Vision-Language-Action (VLA) models have leveraged pre-trained Vision-Language Models (VLMs) to improve the generalization capabilities. VLMs, typically pre-trained on vision-language understanding tasks, provide rich semantic knowledge and reasoning abilities. However, prior research has shown that VLMs often focus on high-level semantic content and neglect low-level features, limiting their ability to capture detailed spatial information and understand physical dynamics. These aspects, which are crucial for embodied control tasks, remain underexplored in existing pre-training paradigms. In this paper, we investigate the training paradigm for VLAs, and introduce \textbf{UP-VLA}, a \textbf{U}nified VLA model training with both multi-modal \textbf{U}nderstanding and future \textbf{P}rediction objectives, enhancing both high-level semantic comprehension and low-level spatial understanding. Experimental results show that UP-VLA achieves a 33% improvement on the Calvin ABC-D benchmark compared to the previous state-of-the-art method. Additionally, UP-VLA demonstrates improved success rates in real-world manipulation tasks, particularly those requiring precise spatial information.

cs.CV↗

arXiv2025

Wide-field Hyperspectral Optical Microscopy for Rapid Characterization of Two-Dimensional Semiconductors and Heterostructures

Electronic and optoelectronic applications of two-dimensional (2D) semiconductors demand precise control over material quality, including thickness, composition, doping, and defect density. Conventional benchmarking methods (e.g., charge transport, confocal mapping, electron or scanning probe microscopy) are slow, perturb sample quality, or involve trade-offs between speed, resolution, and scan area. To accelerate assessment of 2D semiconductors, we demonstrate a broadband, wide-field hyperspectral optical microscope for 2D materials (2D-HOM) that rapidly captures a spatial-spectral data cube within seconds. The data cube includes x-y spatial coordinate (a 300 * 300 $μ\mathrm{m}^2$ field, with ~ 1 $μ\mathrm{m}$ resolution) and a selectable wavelength range between 1100 to 200 nm at each pixel. Using synthesized films and heterostructures of transition metal dichalcogenides ($\mathrm{MoS}_{2}$, $\mathrm{WS}_{2}$, $\mathrm{V}_{x}\mathrm{W}_{1-x}\mathrm{S}_{2}$, and $\mathrm{WSe}_{2}$), we show that this cost-effective technique detects spectral fingerprints of material identity, doping, grain boundaries, and alloy composition, and enables advanced analysis, including unsupervised machine learning for spatial segmentation.

cond-mat.mtrl-sci↗

arXiv2025

Aggregating Inverter-Based Resources for Fast Frequency Response: A Nash Bargaining Game-Based Approach

This paper proposes a multi-objective optimization (MOO) approach for grid-level frequency regulation by aggregating inverter-based resources (IBRs). Virtual power plants (VPPs), acting as aggregators, can efficiently respond to dynamic response requirements from the grid. Through parametric modeling, grid-level frequency regulation requirements are accurately quantified and translated into a feasible parameter region defined by device-level parameters. Based on this feasible region, an MOO model is developed to address the conflicting demands of IBRs during frequency response. A Nash bargaining game-based approach is then employed to optimally allocate regulation requirements within the VPP, balancing the various demands of the IBRs. Numerical experiments demonstrate the effectiveness of the proposed method in enhancing frequency stability and improving coordination among IBRs.

eess.SY↗

arXiv2025

Dynamic Virtual Power Plants with Robust Frequency Regulation Capability

The rapid integration of inverter-based resources (IBRs) into power systems has identified frequency security challenges due to reduced inertia and increased load volatility. This paper proposes a robust power reserve decision-making approach for dynamic virtual power plants (DVPPs) to address these challenges, especially under temporally sequential and uncertain disturbances. An analytical model is developed to characterize the system's frequency response dynamics, enabling the quantification of virtual inertia and virtual damping requirements to meet rate-of-change-of-frequency (RoCoF), frequency nadir, and steady-state deviation constraints. By analytically deriving the regulation power dynamics, the required virtual inertia and damping parameters for the DVPP are determined in a robust way. Then, the total power reserve decision is made by optimally allocating the parameters and calculating the actual power reserves for IBRs, fully considering their economic diversity. Finally, case studies conducted on an IEEE nine-bus system demonstrate the effectiveness of the proposed approach. The results indicate the high reliability of the proposed approach in ensuring frequency security.

eess.SY↗

arXiv2025

GCFL: A Gradient Correction-based Federated Learning Framework for Privacy-preserving CPSS

Federated learning, as a distributed architecture, shows great promise for applications in Cyber-Physical-Social Systems (CPSS). In order to mitigate the privacy risks inherent in CPSS, the integration of differential privacy with federated learning has attracted considerable attention. Existing research mainly focuses on dynamically adjusting the noise added or discarding certain gradients to mitigate the noise introduced by differential privacy. However, these approaches fail to remove the noise that hinders convergence and correct the gradients affected by the noise, which significantly reduces the accuracy of model classification. To overcome these challenges, this paper proposes a novel framework for differentially private federated learning that balances rigorous privacy guarantees with accuracy by introducing a server-side gradient correction mechanism. Specifically, after clients perform gradient clipping and noise perturbation, our framework detects deviations in the noisy local gradients and employs a projection mechanism to correct them, mitigating the negative impact of noise. Simultaneously, gradient projection promotes the alignment of gradients from different clients and guides the model towards convergence to a global optimum. We evaluate our framework on several benchmark datasets, and the experimental results demonstrate that it achieves state-of-the-art performance under the same privacy budget.

cs.LG↗

arXiv2025

Optimal Frequency Support from Virtual Power Plants: Minimal Reserve and Allocation

This paper proposes a novel reserve-minimizing and allocation strategy for virtual power plants (VPPs) to deliver optimal frequency support. The proposed strategy enables VPPs, acting as aggregators for inverter-based resources (IBRs), to provide optimal frequency support economically. The proposed strategy captures time-varying active power injections, reducing the unnecessary redundancy compared to traditional fixed reserve schemes. Reserve requirements for the VPPs are determined based on system frequency response and safety constraints, ensuring efficient grid support. Furthermore, an energy-based allocation model decomposes power injections for each IBR, accounting for their specific limitations. Numerical experiments validate the feasibility of the proposed approach, highlighting significant financial gains for VPPs, especially as system inertia decreases due to higher renewable energy integration.

eess.SY↗

arXiv2025

A Multi-Sensor Fusion Approach for Rapid Orthoimage Generation in Large-Scale UAV Mapping

Rapid generation of large-scale orthoimages from Unmanned Aerial Vehicles (UAVs) has been a long-standing focus of research in the field of aerial mapping. A multi-sensor UAV system, integrating the Global Positioning System (GPS), Inertial Measurement Unit (IMU), 4D millimeter-wave radar and camera, can provide an effective solution to this problem. In this paper, we utilize multi-sensor data to overcome the limitations of conventional orthoimage generation methods in terms of temporal performance, system robustness, and geographic reference accuracy. A prior-pose-optimized feature matching method is introduced to enhance matching speed and accuracy, reducing the number of required features and providing precise references for the Structure from Motion (SfM) process. The proposed method exhibits robustness in low-texture scenes like farmlands, where feature matching is difficult. Experiments show that our approach achieves accurate feature matching orthoimage generation in a short time. The proposed drone system effectively aids in farmland detection and management.

cs.CV↗