arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2609.32505

Fast Differentiable SVD on GPU via Polar Decomposition

Uliana Parkina·Askar Tsyganov·Sergei Kudriashov·Sergey Samsonov·Maxim Rakhuba
View original source ↗Read source document ↗Export metadata ↓

Abstract

We present a fully GPU-oriented SVD pipeline based on polar decomposition, motivated by iterative methods that rely solely on matrix multiplications, such as the Newton-Schulz iteration. We show that this approach enables up to a $2\times$ speedup compared to standard implementations. Furthermore, we derive a numerically stable backward pass for the polar decomposition and leverage it to obtain a fully differentiable SVD. Our methods are released as open-source implementations in both PyTorch and JAX: https://github.com/fallnlove/cans_svd.

Explore related subjects

math.NAcs.LGcs.NA

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Uliana Parkina, Askar Tsyganov, Sergei Kudriashov, Sergey Samsonov, Maxim Rakhuba. 2026-09-26. Fast Differentiable SVD on GPU via Polar Decomposition. https://arxiv.org/abs/2609.32505

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

High order integration of stochastic dynamics on Riemannian manifolds with frozen flow methods

We present a new class of numerical methods for solving stochastic differential equations with additive noise on general Riemannian manifolds with high weak order of accuracy. In opposition to the popular approach with projection methods, the proposed methods are intrinsic: they only rely on geometric operations and avoid coordinates and embeddings. We provide a robust and general convergence analysis and an algebraic formalism of exotic planar Butcher series for the computation of order conditions at any high order. To illustrate the methodology, an explicit method of second weak order is introduced, and several numerical experiments confirm the theoretical findings and extend the approach for the sampling of the invariant measure of Riemannian Langevin dynamics.

math.NA↗
arXiv2026

Convergence analysis of GMRES applied to Helmholtz problems near resonances

The finite element solution of Helmholtz problems near resonant or quasi-resonant frequencies poses significant challenges, as iterative solvers typically suffer from severely degraded convergence. We analyze the convergence behavior of GMRES applied to linear systems arising from such configurations. Theoretical convergence estimates are derived based on harmonic Ritz values, highlighting their proximity to small eigenvalues as a key determining factor. We further examine deflation strategies and their interplay with preconditioning techniques, using the Complex Shifted Laplacian preconditioner as a case study. Numerical experiments on resonant and quasi-resonant test cases validate the theoretical framework and demonstrate the effectiveness of deflation strategies. This study provides new insights and practical guidance for analyzing and improving iterative solvers for time-harmonic problems near resonances.

math.NA↗
arXiv2026

A general framework for the OSRC-preconditioned EFIE in computational electromagnetics

We investigate on-surface radiation condition (OSRC) preconditioners for the electric field integral equation (EFIE), considering Electric-to-Magnetic (EtM) and Magnetic-to-Electric (MtE) approximations applied from both the left and the right. A pseudodifferential analysis establishes the well-posedness of the four resulting formulations and shows that, locally, the OSRC EtM and MtE approximations present a similar structure to the EFIE itself and its inverse, respectively. At the discrete level, both MtE formulations are compatible with the standard RWG/SNC discretization and can be implemented without barycentric refinement, whereas the EtM formulations require an appropriate discrete duality map and BC/RBC spaces. Numerical experiments show that the OSRC preconditioners substantially improve GMRES convergence compared with the unpreconditioned EFIE, with the refinement-free MtE formulations also providing the lowest computational cost among the formulations considered.

math.NA↗