arXiv subjects

Lexing Ying

Publications and source records attributed to Lexing Ying.

At least 19 recordsLinked to original sources

arXiv2026

Multi-Mask Diffusion Language Models for Few-Step Generation

Masked diffusion models (MDMs) are a promising family of language generators, but achieving high-quality few-step generation remains challenging. In MDMs, all forward trajectories collapse to a single fully masked state, leaving no terminal entropy for consistency-style few-step generation. While recent few-step alternatives based on uniform-state diffusion avoid this degeneracy, it becomes harder to distinguish clean tokens from noise than MDMs, which usually harms modeling quality and training efficiency. In this work, we propose a multi-mask diffusion model (MultiMDM) that preserves the masking structure towards few-step generation. In the forward process, each clean token is first pushed towards a designated mask and then gradually mixes over the mask set. As a result, the backward process has a drafting capability by predicting a designated mask before refining to a clean token. We derive a closed-form ELBO training objective for MultiMDM that supports continual training from pretrained MDMs. In addition, we formulate a purely discrete-state consistency distillation scheme, with a shared-Gumbel coupling to reduce pathwise entropy. Experiments on pretraining and distillation show that MultiMDM provides an effective foundation for principled few-step generation.

cs.CL↗

arXiv2026

Quantum Eigenvalue Transformation via Linear Combination of Hamiltonian Simulation: A Weyl Calculus Approach

Linear combination of Hamiltonian simulation (LCHS) provides an efficient method for implementing matrix exponentials $e^{-tA}$ on quantum computers. In this paper, we develop LCHS formulas for computing general matrix functions $f(A)$ when $f$ is analytic on the numerical range of $A$, with $A$ possibly non-normal. The essential technical tool is Weyl calculus, which reduces the construction of LCHS formulas for noncommuting operators to scalar Fourier approximation problems. Our construction yields a quantum eigenvalue transformation algorithm with optimal $\mathcal{O}(\log\frac{1}{\epsilon})$ query complexity scaling. Furthermore, our Weyl-calculus-based theory gives rise to an ansatz-free convex optimization framework that directly produces discrete LCHS formulas. This circumvents the inefficiencies of traditional quadrature rules and yields formulas highly optimized for coherent implementation on quantum computers. In addition, both our theory and optimization framework apply to the simulation of time-dependent dissipative ODE $\frac{\mathrm{d}}{\mathrm{d} t} \psi(t) = -A(t)\psi(t)$, for which we achieve a $2.1\times$ cost reduction over prior art.

quant-ph↗

arXiv2026

Variational inference and density estimation with non-negative tensor of hierarchical tucker format

In this work, we present an efficient method to compress a high-dimensional discrete probability function, i.e., a probability tensor, into a non-negative hierarchical Tucker format. The methodology is a two-stage procedure. In the first stage, we take an existing interpolation method to compress the target tensor into a hierarchical Tucker (HT) in a manner similar to the CUR decomposition for low-rank matrix reconstruction. In the second stage, we fit the first-stage output against a non-negative hierarchical Tucker ansatz using a second-order method tailored specifically for this setting. When the tensor is of order $d$, both stages admit an $\mathcal{O}(d)$ computational complexity, and therefore the proposed methodology readily extends into high-dimensional settings. Numerical experiments show success in compressing various high-dimensional probability tensors.

math.NA↗

arXiv2026

GiVA: Gradient-Informed Bases for Vector-Based Adaptation

As model sizes continue to grow, parameter-efficient fine-tuning has emerged as a powerful alternative to full fine-tuning. While LoRA is widely adopted among these methods, recent research has explored vector-based adaptation methods due to their extreme parameter efficiency. However, these methods typically require substantially higher ranks than LoRA to match its performance, leading to increased training costs. This work introduces GiVA, a gradient-based initialization strategy for vector-based adaptation. It achieves training times comparable to LoRA and maintains the extreme parameter efficiency of vector-based adaptation. We evaluate GiVA across diverse benchmarks, including natural language understanding, natural language generation, and image classification. Experiments show that our approach consistently outperforms or achieves performance competitive with existing vector-based adaptation methods and LoRA while reducing rank requirements by a factor of eight ($8\times$).

cs.CL↗

arXiv2026

Thermal expectation estimation via single-trajectory Gibbs sampling with non-destructive measurements

Estimating thermal expectation values of quantum many-body systems is a central challenge in physics, chemistry, and materials science. Standard quantum Gibbs sampling protocols address this task by preparing the Gibbs state from scratch after every measurement, incurring a full mixing time cost at each step. Recent advances in single-trajectory Gibbs sampling [Jiang et al. 2026] substantially reduce this overhead: once stationarity is reached, measurements can be collected along a single trajectory without re-thermalizing, provided the measurement channel preserves the Gibbs ensemble. However, explicit constructions of such non-destructive measurements have been limited primarily to observables that commute with the Hamiltonian. In this work, we fundamentally extend the single-trajectory framework to arbitrary, non-commuting observables. We provide two measurement constructions that extract measurement information without fully destroying the Gibbs state, thereby eliminating the need for full re-mixing between samples. First, we construct a measurement that satisfies exact detailed balance. This ensures the system remains in equilibrium throughout the trajectory, allowing measurement outcomes to decorrelate in an autocorrelation time that could be significantly shorter than the global mixing time. Second, assuming the underlying quantum Gibbs sampler has a positive spectral gap, we design a simplified measurement scheme that ensures the post-selected state serves as a warm start for rapid re-mixing. This approach successfully decouples the resampling cost from the global mixing time. Both measurement schemes admit efficient quantum circuit implementations, requiring only polylogarithmic Hamiltonian simulation time.

quant-ph↗

arXiv2025

Convergence of the Cumulant Expansion and Polynomial-Time Algorithm for Weakly Interacting Fermions

We propose a randomized algorithm to compute the log-partition function of weakly interacting fermions with polynomial runtime in both the system size and precision. Although weakly interacting fermionic systems are considered tractable for many computational methods such as the diagrammatic quantum Monte Carlo, a mathematically rigorous proof of polynomial runtime has been lacking. In this work we first extend the proof techniques developed in previous works for proving the convergence of the cumulant expansion in periodic systems to the non-periodic case. A key equation used to analyze the sum of connected Feynman diagrams, which we call the tree-determinant expansion, reveals an underlying tree structure in the summation. This enables us to design a new randomized algorithm to compute the log-partition function through importance sampling augmented by belief propagation. This approach differs from the traditional method based on Markov chain Monte Carlo, whose efficiency is hard to guarantee, and enables us to obtain a algorithm with provable polynomial runtime.

quant-ph↗

arXiv2025

Sketch Tomography: Hybridizing Classical Shadow and Matrix Product State

We introduce Sketch Tomography, an efficient procedure for quantum state tomography based on the classical shadow protocol used for quantum observable estimations. The procedure applies to the case where the ground truth quantum state is a matrix product state (MPS). The density matrix of the ground truth state admits a tensor train ansatz as a result of the MPS assumption, and we estimate the tensor components of the ansatz through a series of observable estimations, thus outputting an approximation of the density matrix. The procedure is provably convergent with a sample complexity that scales quadratically in the system size. We conduct extensive numerical experiments to show that the procedure outputs an accurate approximation to the quantum state. For observable estimation tasks involving moderately large subsystems, we show that our procedure gives rise to a more accurate estimation than the classical shadow protocol. We also show that sketch tomography is more accurate in observable estimation than quantum states trained from the maximum likelihood estimation formulation.

quant-ph↗

arXiv2025

Sampling on Metric Graphs

Metric graphs are structures obtained by associating edges in a standard graph with segments of the real line and gluing these segments at the vertices of the graph. The resulting structure has a natural metric that allows for the study of differential operators and stochastic processes on the graph. Brownian motions in these domains have been extensively studied theoretically using their generators. However, less work has been done on practical algorithms for simulating these processes. We introduce the first algorithm for simulating Brownian motions on metric graphs through a timestep splitting Euler-Maruyama-based discretization of their corresponding stochastic differential equation. By applying this scheme to Langevin diffusions on metric graphs, we also obtain the first algorithm for sampling on metric graphs. We provide theoretical guarantees on the number of timestep splittings required for the algorithm to converge to the underlying stochastic process. We also show that the exit probabilities of the simulated particle converge to the vertex-edge jump probabilities of the underlying stochastic differential equation as the timestep goes to zero. Finally, since this method is highly parallelizable, we provide fast, memory-aware implementations of our algorithm in the form of custom CUDA kernels that are up to ~8000x faster than a GPU implementation using PyTorch on simple star metric graphs. Beyond simple star graphs, we benchmark our algorithm on a real cortical vascular network extracted from a DuMuX tissue-perfusion model for tracer transport. Our algorithm is able to run stable simulations with timesteps significantly larger than the stable limit of the finite volume method used in DuMuX while also achieving speedups of up to ~1500x.

math.NA↗

arXiv2025

DriftLite: Lightweight Drift Control for Inference-Time Scaling of Diffusion Models

We study inference-time scaling for diffusion models, where the goal is to adapt a pre-trained model to new target distributions without retraining. Existing guidance-based methods are simple but introduce bias, while particle-based corrections suffer from weight degeneracy and high computational cost. We introduce DriftLite, a lightweight, training-free particle-based approach that steers the inference dynamics on the fly with provably optimal stability control. DriftLite exploits a previously unexplored degree of freedom in the Fokker-Planck equation between the drift and particle potential, and yields two practical instantiations: Variance- and Energy-Controlling Guidance (VCG/ECG) for approximating the optimal drift with minimal overhead. Across Gaussian mixture models, particle systems, and large-scale protein-ligand co-folding problems, DriftLite consistently reduces variance and improves sample quality over pure guidance and sequential Monte Carlo baselines. These results highlight a principled, efficient route toward scalable inference-time adaptation of diffusion models. Our source code is publicly available at https://github.com/yinuoren/DriftLite.

cs.LG↗

arXiv2025

An efficient algorithm for entropic optimal transport under martingale-type constraints

This work introduces novel computational methods for entropic optimal transport (OT) problems under martingale-type conditions. The considered problems include the discrete martingale optimal transport (MOT) problem. Moreover, as the (super-)martingale conditions are equivalent to row-wise (in-)equality constraints on the coupling matrix, our work applies to a prevalent class of OT problems with structural constraints. Inspired by the recent empirical success of Sinkhorn-type algorithms, we propose an entropic formulation for the MOT problem and introduce Sinkhorn-type algorithms with sparse Newton iterations that utilize the (approximate) sparsity of the Hessian matrix of the dual objective. As exact martingale conditions are typically infeasible, we adopt entropic regularization to find an approximate constraint-satisfied solution. We show that, in practice, the proposed algorithms enjoy both super-exponential convergence and robustness with controllable thresholds for total constraint violations.

math.OC↗

arXiv2025

Variational inference and density estimation with non-negative tensor train

This work proposes an efficient numerical approach for compressing a high-dimensional discrete distribution function into a non-negative tensor train (NTT) format. The two settings we consider are variational inference and density estimation, whereby one has access to either the unnormalized analytic formula of the distribution or the samples generated from the distribution. In particular, the compression is done through a two-stage approach. In the first stage, we use existing subroutines to encode the distribution function in a tensor train format. In the second stage, we use an NTT ansatz to fit the obtained tensor train. For the NTT fitting procedure, we use a log barrier term to ensure the positivity of each tensor component, and then utilize a second-order alternating minimization scheme to accelerate convergence. In practice, we observe that the proposed NTT fitting procedure exhibits drastically faster convergence than an alternative multiplicative update method that has been previously proposed. Through challenging numerical experiments, we show that our approach can accurately compress target distribution functions.

math.NA↗

arXiv2025

Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approach

Diffusion models (DMs) have proven to be effective in modeling high-dimensional distributions, leading to their widespread adoption for representing complex priors in Bayesian inverse problems (BIPs). However, current DM-based posterior sampling methods proposed for solving common BIPs rely on heuristic approximations to the generative process. To exploit the generative capability of DMs and avoid the usage of such approximations, we propose an ensemble-based algorithm that performs posterior sampling without the use of heuristic approximations. Our algorithm is motivated by existing works that combine DM-based methods with the sequential Monte Carlo (SMC) method. By examining how the prior evolves through the diffusion process encoded by the pre-trained score function, we derive a modified partial differential equation (PDE) governing the evolution of the corresponding posterior distribution. This PDE includes a modified diffusion term and a reweighting term, which can be simulated via stochastic weighted particle methods. Theoretically, we prove that the error between the true posterior distribution can be bounded in terms of the training error of the pre-trained score function and the number of particles in the ensemble. Empirically, we validate our algorithm on several inverse problems in imaging to show that our method gives more accurate reconstructions compared to existing DM-based methods.

cs.LG↗

arXiv2025

On a spherically lifted spin model at finite temperature

We investigate an $n$-vector model over $k$ sites with generic pairwise interactions and spherical constraints. The model is a lifting of the Ising model whereby the support of the spin is lifted to a hypersphere. We show that the $n$-vector model converges to a limiting distribution at a rate of $n^{-1/2 + o(1)}$. We show that the limiting distribution for $n \to \infty$ is determined by the solution of an equality-constrained maximization task over positive definite matrices. We prove that the obtained maximal value and maximizer, respectively, give rise to the free energy and correlation function of the limiting distribution. In the finite temperature regime, the maximization task is a log-determinant regularization of the semidefinite program (SDP) in the Goemans-Williamson algorithm. Moreover, the inverse temperature determines the regularization strength, with the zero temperature limit converging to the SDP in Goemans-Williamson. Our derivation draws a curious connection between the semidefinite relaxation of integer programming and the spherical lifting of sampling on a hypercube. To the authors' best knowledge, this work is the first to solve the setting of fixed $k$ and infinite $n$ under unstructured pairwise interactions.

math.PR↗

arXiv2025

Inverse nonlinear fast Fourier transform on SU(2) with applications to quantum signal processing

The nonlinear Fourier transform (NLFT) extends the classical Fourier transform by replacing addition with matrix multiplication. While the NLFT on $\mathrm{SU}(1,1)$ has been widely studied, its $\mathrm{SU}(2)$ variant has only recently attracted attention due to emerging applications in quantum signal processing (QSP) and quantum singular value transformation (QSVT). In this paper, we investigate the inverse NLFT on $\mathrm{SU}(2)$ and establish the numerical stability of the layer stripping algorithm for the first time under suitable conditions. Furthermore, we develop a fast and numerically stable algorithm, called inverse nonlinear fast Fourier transform, for performing inverse NLFT with near-linear complexity. This algorithm is applicable to computing phase factors for both QSP and the generalized QSP (GQSP).

quant-ph↗

arXiv2025

Initialization and training of matrix product state probabilistic models

Modeling probability distributions via the wave function of a quantum state is central to quantum-inspired generative modeling and quantum state tomography (QST). We investigate a common failure mode in training randomly initialized matrix product states (MPS) using gradient descent. The results show that the trained MPS models do not accurately predict the strong interactions between boundary sites in periodic spin chain models. In the case of the Born machine algorithm, we further identify a causality trap, where the trained MPS models resemble causal models that ignore the non-local correlations in the true distribution. We propose two complementary strategies to overcome the training failure -- one through optimization and one through initialization. First, we develop a natural gradient descent (NGD) method, which approximately simulates the gradient flow on tensor manifolds and significantly enhances training efficiency. Numerical experiments show that NGD avoids local minima in both Born machines and in general MPS tomography. Remarkably, we show that NGD with line search can converge to the global minimum in only a few iterations. Second, for the BM algorithm, we introduce a warm-start initialization based on the TTNS-Sketch algorithm. We show that gradient descent under a warm initialization does not encounter the causality trap and admits rapid convergence to the ground truth.

math.NA↗

arXiv2025

A Unified Approach to Analysis and Design of Denoising Markov Models

Probabilistic generative models based on measure transport, such as diffusion and flow-based models, are often formulated in the language of Markovian stochastic dynamics, where the choice of the underlying process impacts both algorithmic design choices and theoretical analysis. In this paper, we aim to establish a rigorous mathematical foundation for denoising Markov models, a broad class of generative models that postulate a forward process transitioning from the target distribution to a simple, easy-to-sample distribution, alongside a backward process particularly constructed to enable efficient sampling in the reverse direction. Leveraging deep connections with nonequilibrium statistical mechanics and generalized Doob's $h$-transform, we propose a minimal set of assumptions that ensure: (1) explicit construction of the backward generator, (2) a unified variational objective directly minimizing the measure transport discrepancy, and (3) adaptations of the classical score-matching approach across diverse dynamics. Our framework unifies existing formulations of continuous and discrete diffusion models, identifies the most general form of denoising Markov models under certain regularity assumptions on forward generators, and provides a systematic recipe for designing denoising Markov models driven by arbitrary L\'evy-type processes. We illustrate the versatility and practical effectiveness of our approach through novel denoising Markov models employing geometric Brownian motion and jump processes as forward dynamics, highlighting the framework's potential flexibility and capability in modeling complex distributions.

cs.LG↗

arXiv2025

Solving the Fokker-Planck equation of discretized Dean-Kawasaki models with functional hierarchical tensor

We introduce a novel numerical scheme for solving the Fokker-Planck equation of discretized Dean-Kawasaki models with a functional tensor network ansatz. The Dean-Kawasaki model describes density fluctuations of interacting particle systems, and it is a highly singular stochastic partial differential equation. By performing a finite-volume discretization of the Dean-Kawasaki model, we derive a stochastic differential equation (SDE). To fully characterize the discretized Dean-Kawasaki model, we solve the associated Fokker-Planck equation of the SDE dynamics. In particular, we use a particle-based approach whereby the solution to the Fokker-Planck equation is obtained by performing a series of density estimation tasks from the simulated trajectories, and we use a functional hierarchical tensor model to represent the density. To address the challenge that the sample trajectories are supported on a simplex, we apply a coordinate transformation from the simplex to a Euclidean space by logarithmic parameterization, after which we apply a sketching-based density estimation procedure on the transformed variables. Our approach is general and can be applied to general density estimation tasks over a simplex. We apply the proposed method successfully to the 1D and 2D Dean-Kawasaki models. Moreover, we show that the proposed approach is highly accurate in the presence of external potential and particle interaction.

math.NA↗

arXiv2025

Wavelet-based density sketching with functional hierarchical tensor

We introduce the functional hierarchical tensor under a wavelet basis (FHT-W) ansatz for high-dimensional density estimation in lattice models. Recently, the functional tensor network has emerged as a suitable candidate for density estimation due to its ability to calculate the normalization constant exactly, a defining feature not enjoyed by neural network alternatives such as energy-based models or diffusion models. While current functional tensor network models show good performance for lattice models with weak or moderate couplings, we show that they face significant model capacity constraints when applied to lattice models with strong coupling. To address this issue, this work proposes to perform density estimation on the lattice model under a wavelet transformation. Motivated by the literature on scale separation, we perform iterative wavelet coarsening to separate the lattice model into different scales. Based on this multiscale structure, we design a new functional hierarchical tensor ansatz using a hierarchical tree topology, whereby information on the finer scale is further away from the root node of the tree. Our experiments show that the numerical rank of typical lattice models is significantly lower under appropriate wavelet transformation. Furthermore, we show that our proposed model allows one to model challenging Gaussian field models and Ginzburg-Landau models.

math.NA↗