arXiv Science⌕ Search

arXiv subjects

Na Li

Publications and source records attributed to Na Li.

At least 19 recordsLinked to original sources

Certified Nash Equilibrium Learning for Nonzero-Sum Jump-Diffusion Games: Policy Iteration and Off-Policy Q-Learning

This paper establishes two model-free reinforcement learning methods for infinite-horizon N-player linear-quadratic nonzero-sum stochastic differential games whose drift, Brownian diffusion, and Poisson jump amplitudes depend on all players' controls. Control dependence in the diffusion and jump terms couples the players' simultaneous feedback updates. A verifiable certificate ensures that the partially model-free policy iteration map is contractive on an invariant closed ball centered at an admissible initial feedback profile. All feedback iterates are mean-square stabilizing, and the algorithm achieves local geometric convergence of feedback gains and value matrices to those associated with a closed-loop Nash equilibrium. The term certified refers to a verifiable convergence certificate that guarantees mean-square stability of all policy iterates and local geometric convergence to a closed-loop Nash equilibrium. To implement the policy updates from data, the off-policy Q-learning algorithm reuses the same behavior-policy data across iterations without requiring the system coefficients or jump intensity. Under quantitative persistence of excitation and a common initialization, exact population regression reproduces the policy-iteration sequence. The same local convergence guarantee therefore applies to off-policy Q-learning at the population level. Simulation results demonstrate the effectiveness of the two proposed algorithms.

math.OC↗

Spectral Ghost in Representation Learning: A Unified View from Component Analysis to Self-Supervised Learning

Self-supervised learning (SSL) extracts representations from massive unlabeled data that transfer to a variety of downstream tasks. Its empirical success has produced a dizzying array of objectives and procedures, yet a clear understanding of what representation is desired, and why these seemingly different methods work, is still missing. In this paper, we develop a unified spectral framework for representation learning. We first show that a representation is sufficient for arbitrary downstream prediction with a linear head if it spans the spectral decomposition of the pointwise mutual information, which can be extracted purely from unlabeled data. This view organizes representation learning along two axes, the parametrization of the spectral representation (linear, energy-based, denoising, nonlinear, latent-variable, and multi-modal) and the learning objective for fitting it, under which a broad range of existing methods, including SimCLR, MoCo, BYOL, Barlow Twins, VICReg, DINO, MAE, JEPA, BERT, GPT, and CLIP, as well as classic PCA, CCA, Laplacian eigenmaps, LLE, and SNE, are recovered as particular combinations. The framework also clarifies optimization behaviors: the batch-size requirement is determined by whether the objective is unbiased, a consistent surrogate, or biased, rather than by contrastive vs. non-contrastive losses. Finally, the framework inspires new scalable algorithms and extends spectral representations to causal inference and reinforcement learning.

cs.LG↗

Forward-Value Discretization: Faster Stochastic Diffusion Sampling Without High-Order Approximation

Stochastic diffusion samplers often require substantially more neural function evaluations (NFEs) than deterministic samplers to achieve comparable sample quality. Although higher-order discretization accelerates deterministic sampling, its benefits for stochastic sampling remains unclear. We establish matching upper and worst-case lower bounds showing that stochastic samplers with higher-order approximations remain first-order convergent, even with exact score functions. Thus, increasing the approximation order does not generally improve the convergence order. Motivated by this limitation, we introduce forward-value discretization, a training-free framework that evaluates the data/noise prediction model at a lookahead estimate constructed from previous predictions. For deterministic sampling, the resulting Forward DPMSolver (F-DPMSolver), improves sample quality at low NFE budgets through its DDIM-based variant, despite remaining first-order convergent. For stochastic sampling, we develop Forward DDPM (F-DDPM), which evaluates the model at a cleaner lookahead point adjusted for subsequent noise injection. This estimate reuses both the previous model prediction and the previously injected noise. Both variants admit first-order convergence guarantees with one model evaluation per iteration. Across CIFAR-10, ImageNet, FFHQ, and LSUN, F-DDPM consistently achieves lower FID than the evaluated stochastic baselines at matched NFE budgets. With 1.5 times as many NFEs as deterministic baselines, it achieves competitive sample quality, narrowing the practical efficiency gap between stochastic and deterministic sampling. Code is available at https://github.com/Na-Li66/F-DDPM.

cs.LG↗

Efficient Factorized Volume Synthesis with Lifted Inter-Slice Feature Trajectories

High-resolution 3D medical image generation remains challenging because fully volumetric models are computationally expensive, while efficient 2D slice generators often fail to preserve anatomical consistency across the third dimension. We propose LiFT, a framework for Lifted inter-slice Feature Trajectories that factorizes 3D volume synthesis into per-slice image generation and inter-slice trajectory learning. Rather than using a fully volumetric synthesis backbone, LiFT treats a volume as an ordered trajectory in feature space, capturing how anatomical structures appear, transform, and disappear across depth. We demonstrate the LiFT framework in both unconditional and conditional settings. In unconditional generation, a tri-planar drifting loss matches feature distributions between generated and real volumes across three orthogonal planes; in paired translation, a bidirectional $z$-context mixer trained with paired target supervision supplies through-plane coherence while preserving per-slice fidelity. We evaluate LiFT on BraTS 2023 for unconditional and missing-modality MR synthesis, and on SynthRAD2023 for MR-to-CT translation. LiFT achieves competitive image quality while retaining the efficiency of predominantly 2D synthesis, operating up to $390\times$ faster than a volumetric diffusion baseline and consistently improving through-plane coherence through explicit inter-slice modeling. These results establish lightweight inter-slice trajectory learning as an efficient approach to high-resolution 3D medical synthesis.

cs.CV↗

The Smart Buildings Challenge: Learning Foundation Models for Real-World Optimization at Scale

Buildings cause nearly 40\% of carbon emissions in the US. Of total building emissions, HVAC systems are responsible for approximately 40-60\% and account for 15\% of global energy consumption. Despite advances in artificial intelligence, most buildings still operate as inefficiently as they did decades ago. Building optimization, perhaps more so than any other optimization problem, has enormous potential for Machine Learning (ML) driven impact on climate change, in both the short and long term. One of the biggest obstacles for building optimization is the lack of policy transfer. Most building control systems, whether simulation or learning based, don't generalize: they only work well for the building they were developed for, but not for other buildings. To solve this problem, we propose a challenge for the development of building foundation models that easily adapt to new buildings with minimal configuration and thereby enable building optimization to scale across the globe. Participants will build control policies on a set of simulated buildings, and have their policies evaluated on an unseen set of buildings. The buildings vary in size, layout, weather, material properties, and HVAC system. Successful outcomes from this initiative promise a massive leap forward in global sustainability efforts and demonstrate the great potential of ML to combat climate change.

cs.AI↗

Model-Based Online Decision Making via Generative Trajectory Planning

Online decision-making faces a tension between efficient inference and long-horizon reasoning. Value-based methods rely on short-horizon bootstrapping that limits the propagation of long-term information, while trajectory optimization methods plan explicitly but must solve a high-dimensional optimization problem at every decision step. We propose Generative Trajectory Planning (GTP), a model-based framework that addresses this tension by learning a generative model as a proposal distribution over action sequences. Instead of optimizing sequences from scratch, GTP samples candidate sequences from a diffusion prior and refines them using learned dynamics, reward, and value models, enabling efficient trajectory-level reasoning. Across continuous control and long-horizon manipulation tasks, GTP matches or exceeds state-of-the-art model-free, diffusion-policy, and model-based baselines, and remains stable in regimes where strong baselines degrade. Our results suggest that generative models are best understood not as policies, but as structured proposal distributions for planning, offering a new perspective on how to integrate generative modeling with model-based decision-making.

cs.LG↗

GUI Agents for Continual Game Generation

Generating a game is not the same as making one playable. Existing code-generation approaches often translate a prompt directly into an artifact, leaving interaction-level failures undetected. We argue that game generation requires a player and study two roles for graphical user interface (GUI) agents. First, we introduce \textbf{PlaytestArena}, an evaluation environment containing 200 browser-based game-generation tasks across eight genres, each paired with rubrics of expected in-play behaviors. An independent GUI judge loads and plays each build to adjudicate these rubrics. Second, we propose \textbf{Play2Code}, in which a game agent and a rubric-blind GUI playtester iteratively generate, play, and refine games through shared memory. The playtester provides gameplay traces and actionable feedback, while a separate GPT-5.5 judge assigns final benchmark scores. Across three frontier backbones, Play2Code achieves a 66.8\% rubric pass rate, outperforming single-pass and agentic-coding baselines by 37.1 and 14.6 points, respectively. Its scores also improve monotonically across refinement rounds. Further analysis shows that GUI-agent feedback is fully logged and traceable, while its priorities vary substantially across model backbones. These results establish GUI playtesting as an evaluation and refinement signal for interactive code generation. Our project website is available at https://continual-game-generation.vercel.app/

cs.SE↗

RLE-Bench: A Qualifying Exam for Coding Agents as Robot Learning Engineers

Coding agents are beginning to move beyond purely digital tasks to tackle physical-world challenges, particularly in robotics. Existing robotics benchmarks, however, primarily focus on the performance of individual artifacts, such as policies or controllers, offering limited coverage of coding agents' broader engineering capabilities. Real-world robotics extends beyond control: agents must build, integrate, diagnose, and improve heterogeneous artifacts under resource constraints and reason from multimodal feedback. To evaluate these broader capabilities, we introduce RLE-Bench, a benchmark of robot-learning tasks spanning four representative robotics development workflows: interactive control, policy learning, perception and estimation, and mechanical design. We use diverse task-specific metrics to evaluate the artifacts submitted by the coding agents, from the success rate the agents achieved to the policy agents trained, the harness agent built, and the mechanical structures the agent designed. We aggregate these metrics into an overall RLE Index and report workflow-specific capability profiles, enabling systematic comparison of coding agents' capabilities across multiple capability dimensions. Beyond performance ranks, we also conduct in-depth case studies examining agent behavior on representative tasks, highlighting both current capabilities and limitations, and pointing to the opportunities robotics tasks have to offer for future agent training.

cs.RO↗

D-IMPL: A Diffusion-based Solver for Parameterized BBOs

Diffusion models have demonstrated strong power in generative modeling tasks across multiple domains, exhibiting a remarkable capability of learning complex distributions from samples. In this paper, we leverage such capability to design an efficient universal diffusion-based solver for parameterized black-box optimizations (BBO), where the optimizer has only black-box access to queries of the objective function at the learning stage, yet is able to reduce the additional computational cost at the inference stage for each BBO instance while also capturing the potential multi-modal landscape of non-convex objectives. To cast our formulation as a compatible generative modeling task, we introduce the notion of minimization policy as a new solution concept, which defines a sampling distribution over the solutions that should concentrate around the minimizer set for each BBO instance. We then propose Diffusion-based Iterative Minimization Policy Learning (D-IMPL), a practical generative-model-based solver for solving parameterized BBOs that employs diffusion models to learn a minimization policy, whose density is proportional to the exponential of the negated objective values, thereby amortizing the computational costs across different BBO instances. Furthermore, we demonstrate the performance of our D-IMPL algorithm by establishing a sample complexity guarantee showing that a $δ$-approximate minimization policy can be effectively learned within $O(\log(1/δ))$ iterations, and by extensive empirical evaluations over a range of constrained and unconstrained BBO tasks.

cs.LG↗

One- and two-dimensional cluster states for topological phase simulation and measurement-based quantum computation

Quantum entanglement is a fundamental resource for quantum information processing and serves as a critical benchmark for quantum hardware performance. Cluster states are a special class of entangled states that serve as universal resources for measurement-based quantum computation and possess an intrinsic symmetry-protected topological order, which confers robustness against symmetry-respecting noise. Here we report the scalable preparation and verification of genuine multipartite cluster states on the 105-qubit Zuchongzhi 3.1 superconducting processor. We achieve one-dimensional cluster states of up to 95 qubits and two-dimensional cluster states of up to 72 qubits. The symmetry-protected topological cluster states exhibit input-state-dependent robustness under symmetry-breaking perturbations due to an operational parity structure that enhances the performance of measurement-based quantum computation. Furthermore, we use our two-dimensional cluster states to implement the Deutsch-Jozsa algorithm within the measurement-based quantum computation framework, achieving higher output-state fidelity compared with traditional circuit-based models and a query efficiency advantage over classical approaches. Our work establishes a scalable platform that combines large-scale entanglement generation, symmetry-protected topological order and practical quantum algorithms to enable robust, fault-tolerant measurement-based quantum computation.

quant-ph↗

Colossal reversible conductivity switching by room-temperature oxygen-vacancy ordering in Aurivillius oxide films

Oxygen vacancies are central to the functionality of oxides, yet they typically exist as randomly distributed point defects, limiting the ability to precisely manipulate their collective behavior. Here, we report the room-temperature formation of a long-range-ordered oxygen-vacancy superstructure in single-crystalline Aurivillius-phase Bi2WO6 thin films via a mild nitrogen-plasma treatment. This structural transformation unlocks a colossal, reversible modulation of electrical conductivity by more than nine orders of magnitude, accompanied by a striking optical transition from transparent to black. Atomic-resolution imaging and spectroscopy reveal that the vacancies selectively order within the perovskite-like tungsten oxide layers, forming a coherent defect lattice that is absent in the pristine film. Oxygen-plasma treatment removes the vacancy superstructure and restores the initial state, whereas subsequent nitrogen-plasma treatment reconstructs it, enabling repeatable room-temperature switching between distinct structural, electronic and optical states. The phenomenon is also observed in another Aurivillius member, Bi2MoO6, suggesting its generality across the Aurivillius family. These findings establish a new paradigm for atomic-scale defect engineering - using gentle plasma chemistry to construct ordered defect lattices, opening avenues for reversible property modulation in complex oxides.

cond-mat.mtrl-sci↗

Linear-quadratic mixed Stackelberg-zero-sum game for mean-field regime switching system

Motivated by a green finance problem, a linear-quadratic Stackelberg differential game for a regime switching system involving one leader and two followers is studied. The two followers engage in a zero-sum differential game, and both the state system and the cost functional incorporate a conditional mean-field term. Applying continuation method and induction method, we first establish the existence and uniqueness of the conditional mean-field forward-backward stochastic differential equations with Markovian switching. Based on it, we prove the unique solvability of Hamiltonian systems associated with the two followers and the leader. Moreover, utilizing stochastic maximum principle, decoupling approach and optimal filtering technique, we obtain the optimal feedback strategies for the two followers and the leader. Employing the theoretical results, we solve the green finance problem with some numerical simulations.

math.OC↗

Scaling Muon for Diffusion Transformers

The matrix-aware optimizer Muon improves large model training by balancing updates across singular directions, yet its scaling behavior and end-to-end efficiency on large Diffusion Transformers (DiTs) remain unclear. We first establish Muon's scaling behavior on DiTs from 1.3B to 15B parameters, showing that its optimization and generative quality advantages over AdamW persist across model scales. However, at scale, the 5-step Newton--Schulz iteration (NS5) performed at every optimization step, together with full-momentum materialization, introduces substantial computation and communication overhead that can offset Muon's step-efficiency advantage. We introduce \emph{Periodic Row-wise Muon}, which performs a full NS5 spectral update once every \(K\) steps and applies a low compute and communication cost row-wise constrained update based on the current momentum at the remaining steps. We further co-design a distributed implementation that operates directly on sharded momentum during non-refresh steps and accelerates spectral refreshes through bucketed all-gather and communication--computation overlap. Across all scales, Muon improves the best observed generative quality over AdamW by 12.9--19.1\%. Compared with vanilla Muon, Periodic Row-wise Muon remains within 0.5\% in best generative quality on the 1.3B--4B models and improves it by 4.5\% at 9B. It reduces optimizer time by 46.9--54.3\%, end-to-end step time by 15.7--24.3\%, and logical communication volume by 66.7\%, while reaching its respective best generative quality with 33.7--64.8\% less active training time. These results show that Periodic Row-wise Muon preserves Muon's generative quality advantage while translating it into end-to-end training efficiency for large DiTs.

cs.LG↗

ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings

Recent advances in continuous diffusion and flow-based language models (LMs) have achieved performance competitive with discrete LMs. However, existing continuous frameworks still rely on decoders supervised with cross entropy (CE) because the flow trajectories are not guaranteed to terminate at valid token embeddings. Motivated by this limitation, we introduce \textbf{ConvergeFlow}, an embedding-space flow-based LM, which constrains the data predictor to the convex hull of token embeddings and trains it solely with the mean squared error objective induced by flow matching. Under suitable regularity conditions, we prove that the resulting flow converges to valid token embeddings despite errors in the data predictor, enabling direct token prediction without a CE-supervised decoder. We further develop three sampling mechanisms for controlling the trade-off between the generative perplexity and entropy. Experiments on OpenWebText demonstrate that ConvergeFlow achieves performance competitive with existing continuous and discrete diffusion LMs. These findings demonstrate the potential of the flow-based paradigm for language modeling. Our code is available at https://github.com/Na-Li66/ConvergeFlow.

cs.CL↗

High fidelity control of superconducting qubits with optical transmitted signal

Superconducting circuits exhibit remarkable potential for constructing large-scale quantum simulation and computation systems, featuring numerous qubits, extended coherence time, and precise control. Nevertheless, the growing number of signal cables poses a challenge in dilution refrigerators due to space and heat load constraints. To overcome this issue, we experimentally implemented an optically-assisted transmission line as an alternative to coaxial cables. By modulating microwave signals on laser intensities at room temperature and regenerating the signals at a cryogenic plate within the dilution refrigerator, we demonstrated full control of superconducting qubits using photocurrent. We demonstrate and benchmark both single-qubit and two-qubit gates on frequency tunable transmon qubits, achieving fidelities of 99.915% $\pm$ 0.005% and 99.676% $\pm$ 0.041%, respectively, which have reached the requirement of the surface code.

quant-ph↗

Q-Regularized Generative Auto-Bidding: From Suboptimal Trajectories to Optimal Policies

With the rapid development of e-commerce, auto-bidding has become a key asset in optimizing advertising performance under diverse advertiser environments. The current approaches focus on reinforcement learning (RL) and generative models. These efforts imitate offline historical behaviors by utilizing a complex structure with expensive hyperparameter tuning. The suboptimal trajectories further exacerbate the difficulty of policy learning. To address these challenges, we proposes QGA, a novel Q-value regularized Generative Auto-bidding method. In QGA, we propose to plug a Q-value regularization with double Q-learning strategy into the Decision Transformer backbone. This design enables joint optimization of policy imitation and action-value maximization, allowing the learned bidding policy to both leverage experience from the dataset and alleviate the adverse impact of the suboptimal trajectories. Furthermore, to safely explore the policy space beyond the data distribution, we propose a Q-value guided dual-exploration mechanism, in which the DT model is conditioned on multiple return-to-go targets and locally perturbed actions. This entire exploration process is dynamically guided by the aforementioned Q-value module, which provides principled evaluation for each candidate action. Experiments on public benchmarks and simulation environments demonstrate that QGA consistently achieves superior or highly competitive results compared to existing alternatives. Notably, in large-scale real-world A/B testing, QGA achieves a 3.27% increase in Ad GMV and a 2.49% improvement in Ad ROI.

cs.LG↗

Indefinite Stochastic Linear-Quadratic Optimal Control Problem for a Markov Regime-Switching Model

This paper investigates an indefinite stochastic linear-quadratic (SLQ) control problem with parameters subject to Markov regime-switching. Based on the well-posedness of the SLQ problem, we introduce a relaxed compensator that extends SLQ control problems from the positive definite case to the indefinite case. We analyze the corresponding stochastic Hamiltonian system for both unconstrained and constrained control cases under the indefinite framework and derive the corresponding optimal open-loop controls. We further investigate the associated Riccati equations for both unconstrained and constrained control cases and derive the closed-loop feedback forms of optimal controls. We illustrate the theoretical results with an equity-bond asset allocation problem under un-constrained and non-negative control constraints. Numerical simulations validate the effectiveness of the theoretical framework and demonstrate its practical value in solving complex stochastic control problems with Markov regime-switching.

math.OC↗