arXiv · 2611.19518
Equilibrium measures for anisotropic exponential interaction energies with applications to transformers
Abstract
We study the minimization of anisotropic exponential interaction energies over probability measures on the sphere, motivated by mean-field models of self-attention in transformer architectures. For every symmetric positive definite interaction matrix, we relate the minimizer to a corresponding anisotropic Brownian motion and use the Feynman--Kac formula and elliptic regularity theory to show that it admits a smooth, strictly positive density on the sphere. We further show that it is the only stationary measure with full support for the dynamics considered. We use the stochastic viewpoint to derive an algorithm for sampling from the minimizer. In dimension two, we also propose a numerical scheme for approximating its density based on a Mathieu series expansion.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Oleksii Kachaiev, Martin Burger, Samira Kabri, Yury Korolev, Tim Roith. 2026-11-14. Equilibrium measures for anisotropic exponential interaction energies with applications to transformers. https://arxiv.science/abs/2611.19518
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.