arXiv subjects

Xuan Zhao

Publications and source records attributed to Xuan Zhao.

At least 19 recordsLinked to original sources

arXiv2026

AgentGUI: An Interface for Observing and Steering Long-Running AI Agents

AI agents are increasingly adept at tackling complex, long-running tasks. With the rapid surge of autonomous capabilities, human oversight is systematically lagging behind due to limited human-centered interfacing. Aiming to address this, we introduce AgentGUI, a user-friendly, locally hosted GUI for seamlessly observing and steering AI agents amid multiple concurrent, long-running sessions. AgentGUI features 1) rich agent trajectory visualizations, 2) effective manual and automated steering, and 3) integration with and coordination between open-source and frontier agent frameworks. A controlled user study demonstrates statistically significant reduction in the time it takes to identify key elements from agent traces (38% faster, p = 0.023). In a preliminary experiment, AgentGUI's automated drift prevention feature raises the task completion rate of small local agents by as high as 34pp across a 0.8B--9B model ladder (N=50 runs per model). AgentGUI is publicly available through its project website (https://agent-gui-project.github.io) and open-source repository (https://github.com/eth-medical-ai-lab/agent-gui), along with a demo video (https://youtube.com/watch?v=GSDyxN1gTF0).

cs.CL↗

arXiv2026

LapaFM: A Laparoscopic Foundation Model via Multi-Stage Continual Learning

Artificial intelligence-based analysis of intraoperative laparoscopic videos has the potential to improve surgical decision-making and precision. However, effective surgical assistance requires models that can generalize across diverse intraoperative tasks. Here we present LapaFM, a foundation model for laparoscopic surgery trained on more than 20,000 minutes of surgical videos using a multi-stage continual learning strategy that mitigates catastrophic forgetting and inter-task interference. LapaFM learns multimodal visual representations and supports multiple tasks, including smoke removal, surgical phase recognition, instrument and polyp segmentation, and lesion identification. The model was evaluated on 13 retrospective and two prospective cohorts collected from different imaging devices. LapaFM consistently outperformed task-specific models and existing foundation models across datasets, and also generalized to gastrointestinal endoscopy tasks, including polyp detection and classification. Notably, it recovered over 90% of smoke-obscured frames and achieved clinician-level or superior performance in lymph node metastasis prediction and prospective polyp classification.

cs.CV↗

arXiv2026

Several Accelerated and Stable Pseudo-Energy-Dissipative Lagrange Multiplier Methods based on Second-order Flow and Variable Splitting

Based on second-order inertial dynamics, we develop two accelerated Lagrange multiplier (LM)-based optimization methods: Linear LM-based second-order flow method and linear LM-based variable and operator splitting method for unconstrained non-convex optimization problems. Relaxed and adaptive variants of both methods are proposed to prevent degeneration of the Lagrange multiplier and to enable automatic adjustment of the time step size. Within the novel second-order LM-based optimization framework, the associated pseudo-energy functional is shown to dissipate monotonically along the iterative trajectory, and convergence of the generated iterates to stationary points is rigorously established. Extensive numerical experiments on function optimization and partial differential equation benchmark problems, including applications as optimizers for physics-informed neural networks (PINNs) and deep operator networks (DeepONets), demonstrate that the proposed methods effectively escape local minima, achieve high accuracy, and exhibit strong robustness with respect to the choice of the initial learning rate during neural network training.

math.OC↗

arXiv2026

Towards an AI Software Factory for Data Systems

AI-assisted coding tools deliver significant acceleration of coding, but only limited impact across the end-to-end software development lifecycle (SDLC)--an Amdahl's law effect! In this paper, we discuss our progress towards building an AI SW Factory that accelerates all the stages of SDLC-Targeting, Coding, Reviewing, and Ops. The AI SW Factory produces a metadata exhaust that enables self-improvement by fine-tuning model weights and updating our World Model (a rich data substrate). We focus on Data Systems and the important class of Evolutionary Coding Tasks (i.e., those with a measurable objective to hill-climb) and report on 1) scaled deployments at Microsoft (tens of repositories) leading to 3x engineering efficiency above agentic coding and up to 22x token efficiency, and 2) several open challenges.

cs.AI↗

arXiv2026

CoLMIN: LLM-based Multi-Decision Path Negotiation for Cooperative Autonomous Driving

Multi-vehicle cooperative autonomous driving enhances the safety and reliability of autonomous driving systems through information sharing among connected vehicles, demonstrating significant potential for improving traffic safety. LLM-based approaches leverage strong reasoning capabilities of LLMs to enable effective inter-vehicle negotiation and improve cooperative driving performance. However, driving decisions in complex traffic scenarios are inherently multi-solution in nature. As a result, existing negotiation-based methods often converge prematurely to suboptimal solutions, hindering consensus formation and limiting the practical deployment of cooperative autonomous driving systems. To address this challenge, we propose CoLMIN, the LLM-based multi-decision path negotiation framework for cooperative autonomous driving, achieving stable decision consensus through multi-decision path negotiation and reflective reasoning. To achieve stable and high-quality consensus in cooperative autonomous driving, CoLMIN consists of three key components: (i) an LLM-based Multi-Intent Negotiation module (LMin), which adopts a Negotiator-Evaluator paradigm and generates multiple candidate driving intentions for joint evaluation; (ii) an Evaluation-based Shallow Reflection Module (ESRM), which analyzes negotiation outcomes and provides feedback to guide subsequent negotiations, thereby accelerating consensus formation; and (iii) an LLM-based Deep Reflection Module (LDRM), which performs long-term reflection over negotiation histories to mitigate cognitive fixation and prevent the system from converging to suboptimal solutions. Experimental results in the CARLA simulation environment demonstrate that CoLMIN significantly outperforms existing methods in challenging interactive driving scenarios.

cs.RO↗

arXiv2026

AnyBox: Efficient Zero-Shot 9DoF Pose Estimation of Boxes for Robotic Manipulation

Recovering the 9D pose of objects, both their 6D pose and 3D dimensions, under clutter and occlusion is a core requirement for warehouse automation, logistics, and manufacturing. Model-based methods are accurate but assume an instance-specific CAD model for every object, which is costly to maintain as inventories change. Model-free and category-level methods relax this assumption, yet they remain vulnerable to the symmetry, weak texture, and heavy occlusion that characterize stacked storage boxes, and they ignore the strong structural priors such scenes provide. We present \textbf{AnyBox}, an efficient zero-shot framework that exploits the geometric regularity of boxes to jointly recover pose and dimensions from a single RGB-D observation. Starting from a canonical category template, AnyBox alternates between pose and scale estimation, using the discrepancy between the reprojected template and the observed mask to drive a binary search over box dimensions. Two lightweight components make this practical: a depth-consistency filter that rejects the implausible hypotheses induced by box symmetry, and an early-stopping rule that replaces the remaining search with a single closed-form update. On public benchmarks and an in-house warehouse dataset, AnyBox improves detection AP by up to 36 points, more than doubling the previous best, and approaches instance-level pipelines that have access to ground-truth CAD models. These gains transfer downstream, raising success by 28\% on a cluttered robotic box-shelving task.

cs.CV↗

arXiv2026

Wafer-scale monolithic integration of Ce:YIG films and magneto-optical isolators on silicon

Silicon integrated cerium doped yttrium iron garnet (Ce:YIG) thin films are promising candidates for integrated nonreciprocal photonic devices, cryogenic photonic modulators and optical computing applications. However, previously reported Ce:YIG thin film on silicon is limited to milimeter sizes. Wafer-scale integration and non-destructive characterization of high quality Ce:YIG thin films on silicon has been elusive. Here, we report growth of 4-inch wafer-scale Ce:YIG thin films on silicon substrates by radio-frequency magnetron sputtering. Strong Faraday effect of 2318 deg/cm, low propagation loss of 80 dB/cm and excellent thickness uniformity of 3.5% is demonstrated across the 4-inch silicon wafer. Furthermore, a custom designed wafer-scale, non-destructive magneto-ellipsometry was established to characterize the film thickness, optical constants and magneto-optical constants across the wafer. Wafer-scale integration of ring resonator type magneto-optical isolators are also demonstrated. Our work demonstrates a step forward toward wafer-scale heterogeneous integration and characterization of magneto-optical thin films on silicon, providing material candidates for non-reciprocal photonic device arrays, magneto-optical in-memory computing networks and integrated magneto-optic magnetometers.

physics.optics↗

arXiv2026

From Scalars to Time Series: Rethinking Implicit Neural Representations for Time-Varying Volumetric Data

Implicit neural representations (INRs) for time-varying volumetric data are typically trained using dense sampling over spatiotemporal coordinates, where each observation corresponds to a single point in space and time. This coordinate-wise formulation requires extensive sampling during optimization, leading to high computational cost and inefficient use of temporal structure. In this work, we revisit this design choice and show that dense spatiotemporal sampling is not necessary for learning time-varying fields. Instead, we represent the data as a collection of spatially indexed time series and train INRs using sequence-level supervision over each spatial location, rather than coordinate-wise scalar samples. This reformulation eliminates the need for dense spatiotemporal sampling and instead learns each spatial location from its full temporal evolution in a structured manner. We demonstrate that this representation is compatible with a range of existing INR architectures and consistently improves reconstruction quality, while significantly reducing training cost. Furthermore, we show that this formulation can be combined with mixture-of-experts architectures, and that our MoE instantiation further improves reconstruction quality compared to both the base reformulation and existing MoE-based INR methods, providing a stronger capacity allocation under heterogeneous temporal dynamics.

cs.AI↗

arXiv2026

Localized crystallization of Ce:YIG thin films on Si using CO2 laser annealing for integrated nonreciprocal photonic device applications

Laser annealing (LA) technique has emerged as an effective method for localized crystallization of magneto-optical (MO) garnet thin films on semiconductor substrates. However, no studies have explored the crystallization and magneto-optical (MO) properties of cerium-substituted yttrium iron garnet (Ce:YIG, Ce1Y2Fe5O12) thin films for integrated photonic device applications using LA technique. In this study, we provide a comprehensive investigation into the laser annealing of Ce:YIG films deposited on SiO2 substrates and silicon nitride photonic waveguides for integrated nonreciprocal photonic device applications. Garnet phase was successfully observed in films grown on SiO2 substrates, and SiN waveguides with laser annealing of sputtered Ce:YIG films on top of a laser annealed Y3Fe5O12 seed layer. The magneto-optical (MO) properties of Ce:YIG films on oxidized Si substrates were found to be comparable to those prepared by rapid thermal annealing (RTA). A Mach-Zehnder Interferometer (MZI) type optical isolator based on Ce:YIG film on SiN was fabricated, exhibiting a saturation Faraday rotation of -2317.7 deg/cm and propagation loss of 188.2 dB/cm. Isolation ratio of 27.1 dB and insertion loss of 10.1 dB were achieved at 1552.7 nm wavelength.

cond-mat.mtrl-sci↗

arXiv2026

Parametric Skills

Since intelligence fundamentally relies on efficient skill acquisition (Chollet, 2019), the ability to leverage skills is critical. For LLMs, skills, manually authored or extracted from task trajectories, are textual recipes encoding mature problem-solving experience and are critical to agentic capabilities. Despite widespread deployment, their utility is limited by the model's ability to comprehend and follow skill instructions, especially under complex and long-context scenarios, where key instructions are difficult to locate and adhere to. To address this limitation, we propose ParametricSkills, a framework that can convert free-form textual skills into parameters at test time, enabling context-free skill exploitation. Specifically, we first construct a large-scale, high-quality skill library, and synthesize single-turn and multi-turn skill exploitation trajectories built around these skills with OpenCode. Using these data, we then train a hypernetwork that parameterizes both the skill content and the test-time exploitation methodology by receiving textual skills and converting them into LoRA adapters. Experimental results on six complex software engineering (SWE) subtasks demonstrate that, the proposed ParametricSkills averagely outperforms in-context learning by 6.44 points as judged by DeepSeek-V4-Flash, while also achieving significantly higher BERT Score and F1 score, confirming its effectiveness. Beyond performance, we further find that parametric skills, being inherently accumulative, offer a preliminary yet promising avenue toward test-time continual learning.

cs.CL↗

arXiv2026

RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data

Counterfactual explanations (CFs) help understand machine learning models by identifying minimal input changes that would lead to alternative model outcomes. Recent work demonstrates their utility for reconstructing black-box models, enabling third-party auditing of opaque decision systems for fairness and accountability. Still, CF-based reconstruction may suffer from decision boundary shifts, overfitting, and restrictive assumptions requiring online query access to target platforms. We propose REconstruction via Counterfactual-Aware waSserstein opTimization (RECAST) under limited data and restricted access, a behavioral surrogate model based on Wasserstein barycentric prototypes. Our approach addresses decision boundary shifts by incorporating CFs as informative, though less representative, samples for both classes, maintaining high surrogate fidelity in low-sample regimes without requiring online access during reconstruction. To enhance fairness auditing, our method enables systematic group fairness diagnostics. Experiments on real-world datasets and various setups show that RECAST effectively achieves high fidelity and query efficiency, as well as stable results even when the access is limited and noisy.

cs.LG↗

arXiv2026

Residual-Space Evolutionary Optimization via Flow-based Generative Models

Data editing with generative methods typically requires differentiable objectives and gradient-based search. However, these assumptions break down in flow-based settings, where edits are performed through forward and backward integration and often involve non-differentiable or black-box objectives. We introduce residual-space evolutionary optimization, a model-agnostic framework that addresses this gap by combining flow-based generative editing with evolutionary algorithms. Building on the observation that conditional flow matching (CFM) can disentangle condition-controlled factors from instance-specific residuals, our framework directly operates in residual space and separates two complementary search regimes: self-pollination performs local exploitation through feature-preserving residual refinement, and cross-pollination promotes broader exploration by recombining residuals across heterogeneous samples. As a proof of concept, we validate on MorphoMNIST, a benchmark dataset for counterfactual generation, and on crystal data, demonstrating that this exploration--exploitation decomposition provides a useful mechanism for balancing target alignment, instance preservation, and diversity, and extends beyond images to real-world scientific domains.

cs.AI↗

arXiv2026

Analytical solution and Lie algebra of the relativistic Boltzmann equation

In this work, we present a novel and more efficient approach to constructing the relativistic Bobylev, Krook, and Wu (BKW) solution [A. V. Bobylev, Sov. Phys. Dokl. 20, 820 (1976); M. Krook and T. T. Wu, Phys. Fluids 20, 1589 (1977)] of the relativistic Boltzmann equation. Introducing a new ansatz for the distribution function, we demonstrate that within this specific ansatz space, only the equilibrium and BKW-type forms yield exact solutions to the nonlinear Boltzmann equation. Furthermore, we also derive the Lie algebra of invariant transformations admitted by the relativistic Boltzmann equation and show that the corresponding symmetry group transformations can be systematically constructed from this algebra.

nucl-th↗

arXiv2026

Counterfactual Transport Flows for Offline Conservative Trajectory Refinement

Offline reinforcement learning (RL) offers a path to policy improvement from logged data alone, using historical returns or other measurable outcomes as world feedback. A key difficulty is improving observed behavior without extrapolating beyond what the offline data supports. We propose \emph{counterfactual transport flows}, a source-conditioned trajectory refinement framework for offline decision-making guided by world feedback. Given a low-feedback candidate trajectory, we construct local preference pairs from offline data by retrieving nearby trajectories in latent trajectory space with higher task-specific feedback, and use them as weak supervision for conservative refinement. The framework learns instance-specific refinement directions: at inference time, a refinement strength parameter controls how far the candidate trajectory is transported, enabling a trade-off between preserving the original behavior and applying stronger improvement. Experiments on D4RL benchmarks, including AntMaze and MuJoCo tasks, show that our method improves behavior from historical returns as world feedback, while providing interpretable trajectory-level refinement paths.

cs.LG↗

arXiv2026

Retarded Correlators of Charge Transport in a Magnetic Field

We study charge transport in a magnetized relativistic plasma using kinetic theory within the relaxation-time approximation. By exactly solving the linearized Boltzmann equation in a uniform magnetic field, we obtain an analytic solution for the distribution function in terms of Bessel functions. Using this solution, we compute the full set of retarded current-current correlators and verify the Ward identities. In the hydrodynamic limit, we extract the charge diffusion modes, demonstrating that the transverse diffusion coefficient is strongly suppressed by the magnetic field, scaling as $1/B_0^2$ in the strong-field regime, while the longitudinal diffusion remains unaffected. Furthermore, we analyze the non-hydrodynamic branch cuts in the complex frequency plane, determining their kinematic thresholds and identifying the underlying wave-particle interactions as longitudinal Landau damping and transverse cyclotron damping.

hep-ph↗

arXiv2026

SteerFace: Debiasing Synthetic Face Generation via Adaptive Residue Perturbation

The shortage of legally compliant data for face recognition training has sparked growing interest in using synthetic data as an alternative. While recent diffusion-based methods enable the generation of photorealistic face images with strong identity adherence and data diversity, their downstream recognition performance still exhibits a significant synthetic-real gap. This paper identifies visual tendency as a previously underexplored limitation, whereby synthetic data exhibit an unrealistic prevalence of visual attributes and thus deviate from the real-data distribution. Visual tendency can be attributed to the generator's conditioning on identity embeddings, through which co-occurring residual visual cues are unintentionally absorbed into learned identity semantics. To discourage the generator from exploiting such visual cues, this paper proposes SteerFace, a simple and efficient training framework that perturbs identity embeddings by steering them toward random orthogonal directions on the embedding hypersphere. The perturbation serves as an identity-preserving regularizer that penalizes the generator's reliance on non-identity components, as supported by theoretical analysis. This paper further introduces an adaptive strategy that learns perturbation strengths with both sample-wise preference and favorable overall statistics. Extensive experiments show that SteerFace effectively mitigates visual tendency, outperforms prior methods in downstream face recognition, and generalizes well across different training datasets and generation pipelines.

cs.CV↗

arXiv2026

Libra: Training the Environment for Agentic Information Retrieval

Information localization within massive repositories is a cornerstone of agentic LLM systems. While synthetic data-driven optimization has proven successful in training LLMs, little attention has been paid to optimizing the agent's working environment (the repository itself) in a data-driven manner. To bridge this gap, we present Libra, a self-evolving framework that introduces mutable "catalogs" (hierarchical Markdown files serving as navigable indices) into the repository. Libra runs an LLM-driven optimization loop where a Prompter generates synthetic queries, a frozen Solver attempts to resolve them by navigating the catalogs, and a Healer rewrites the catalogs in response to the Solver's localization failures. Evaluations across 12 SWE-bench Lite repositories demonstrate that this environmental healing yields continual, logarithmic improvements in code localization accuracy. Furthermore, these environmental improvements transfer zero-shot across different LLMs and problem sets. Although the focus of this paper is to study the general behavior of such a system, we also demonstrate that a minimalist coding agent equipped with Libra-optimized catalogs outperforms state-of-the-art baselines. Code is available at https://github.com/salesforce-misc/Libra and data at https://huggingface.co/datasets/Salesforce/Libra.

cs.IR↗

arXiv2026

Stationary States for Fermions in an External Electric Field

We present a relativistic analysis of fermions in an external electric field by non-perturbatively solving the Dirac equation with a static gauge. Different from the magnetic field effect, the fermion wave function in an electric field oscillates asymptotically, which results in the absence of bound states in an infinite system. For a confined fermion, the confinement is gradually canceled by the electric field, and the fermion becomes deconfined when the electric coupling is stronger than the confinement coupling. However, a fermion in an electric field can be confined to a finite system by applying the MIT bag boundary condition, namely, the disappearing normal component of the probability current at the boundary. The solutions obtained can serve as a basis for calculating dynamical processes in the presence of a strong electric field, such as those occurring in the early stage of relativistic heavy-ion collisions, where an extremely strong electric field is expected to be generated.

nucl-th↗