arXiv ScienceSearch

arXiv subjects

Garrett Mulcahy

Publications and source records attributed to Garrett Mulcahy.

6 recordsLinked to original sources

Diffusion Approximations to Schrödinger Bridges and the Convergence of Entropic Potentials

Consider the Monge-Kantorovich optimal transport problem between two Euclidean densities $μ$ and $ν$ and quadratic cost. The $\varepsilon$-Schrödinger bridge is the solution to the entropic regularized problem with regularization parameter $\varepsilon$. By taking the conditional expectation of the second coordinate given the first under this coupling, we obtain the $\varepsilon$-entropic Brenier map. As $\varepsilon$ goes down to zero, it is known that the entropic Brenier map converges to the quadratic cost optimal transport map between the two measures, i.e., the Brenier map. We show that, under some smoothness and log-concavity constraints on the marginals, the difference between the entropic Brenier map and the Brenier map is equal to $\varepsilon$ times one-half of the score function of the first marginal plus an error that is $o(\varepsilon)$ in $\mathbf{L}^2(μ)$. This expansion holds irrespective of the second marginal $ν$. The proof relies on an approximation of the Schrödinger bridge by a noisy version of the McCann interpolation. Under additional assumptions we show a second approximation to the Schrödinger bridge via so-called Mirror Langevin diffusions, which are Langevin diffusions on the Hessian manifold generated by the Brenier map. These two approximations, that appear quite different at first glance, are nonetheless shown to be closely connected. Our proofs utilize a random surface and a novel stochastic operation called the tangent Markov projection.

math.PR

Noising-Denoising by Large Temperature Schrödinger Bridges

In this note, we establish a connection between denoising diffusion models and large temperature dynamic Schrödinger bridges computed with respect to certain reference processes. In particular, we show that in the large temperature regime (i.e. $T \to +\infty$), the Schrödinger bridge behaves like a forward time diffusion process over the time interval $[0,T/2]$, and approximately like the time reversal of a diffusion process over $[T/2,T]$. This construction provides a process-level characterization of the gradient flow approximation developed by Clerc et al. (2023).

math.PR

Langevin Diffusion Approximation to Same Marginal Schrödinger Bridge

We introduce a novel approximation to the same marginal Schrödinger bridge using the Langevin diffusion. As $\varepsilon \downarrow 0$, it is known that the barycentric projection (also known as the entropic Brenier map) of the Schrödinger bridge converges to the Brenier map, which is the identity. Our diffusion approximation is leveraged to show that, under suitable assumptions, the difference between the two is $\varepsilon$ times the gradient of the marginal log density (i.e., the score function), in $\mathbf{L}^2$. More generally, we show that the family of Markov operators, indexed by $\varepsilon > 0$, derived from integrating test functions against the conditional density of the static Schrödinger bridge at temperature $\varepsilon$, admits a derivative at $\varepsilon=0$ given by the generator of the Langevin semigroup. Hence, these operators satisfy an approximate semigroup property at low temperatures.

math.PR

Diffusion Approximations to Schrödinger Bridges on Manifolds

We present a collection of explicit diffusion approximations to small temperature Schrödinger bridges on manifolds. Our most precise results are when both marginals are the same and the Schrödinger bridge is on a manifold with a reference process given by a reversible diffusion. In the special case that the reference process is the manifold Brownian motion, we use the small time heat kernel asymptotics to show that the gradient of the corresponding Schrödinger potential converges in $L^2$, as the temperature vanishes, to a manifold analogue of the score function of the marginal. As an application of the previous result we show that the Euclidean Schrödinger bridge, computed for the quadratic cost, between two different marginal distributions can be approximated by a transformation of a two point distribution of a stationary Mirror Langevin diffusion.

math.PR

Iterated Schrödinger bridge approximation to Wasserstein Gradient Flows

We introduce a novel discretization scheme for Wasserstein gradient flows that involves successively computing Schrödinger bridges with the same marginals. This is different from both the forward/geodesic approximation and the backward/Jordan-Kinderlehrer-Otto (JKO) approximations. The proposed scheme has two advantages: one, it avoids the use of the score function, and, two, it is amenable to particle-based approximations using the Sinkhorn algorithm. Our proof hinges upon showing that relative entropy between the Schrödinger bridge with the same marginals at temperature $ε$ and the joint distribution of a stationary Langevin diffusion at times zero and $ε$ is of the order $o(ε^2)$ with an explicit dependence given by Fisher information. Owing to this inequality, we can show, using a triangular approximation argument, that the interpolated iterated application of the Schrödinger bridge approximation converge to the Wasserstein gradient flow, for a class of gradient flows, including the heat flow. The results also provide a probabilistic and rigorous framework for the convergence of the self-attention mechanisms in transformer networks to the solutions of heat flows, first observed in the inspiring work SABP22 in machine learning research.

math.PR

Malnormal matrices

We exhibit an operator norm bounded, infinite sequence $\{A_n\}$ of $3n \times 3n$ complex matrices for which the commutator map $X\mapsto XA_n - A_nX$ is uniformly bounded below as an operator over the space of trace-zero self-adjoint matrices equipped with Hilbert--Schmidt norm. The construction is based on families of quantum expanders. We give several potential applications of these matrices to the study of quantum expanders. We formulate several natural conjectures and problems related to such matrices and provide numerical evidence.

math.FA