arXiv subjects

Jan Quan

Publications and source records attributed to Jan Quan.

8 recordsLinked to original sources

arXiv2026

Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives

In this work, we develop proximal preconditioned gradient methods with a focus on spectral gradient methods providing a proximal extension to the Muon and Scion optimizers. We introduce a family of stochastic algorithms that can handle a wide variety of convex and nonconvex constraints and study its convergence under heavy-tailed noise, through a novel analysis tailored to the geometry of the proposed methods. We further propose a variance-reduced version, which achieves faster convergence under standard noise assumptions. Finally, we show that the polynomial iterations used in Muon are more accurately captured by a nonlinear preconditioner than by the ideal matrix sign, leading to a convergence analysis that more faithfully reflects practical implementations.

math.OC↗

arXiv2025

Nonlinearly preconditioned gradient flows

We study a continuous-time dynamical system which arises as the limit of a broad class of nonlinearly preconditioned gradient methods. Under mild assumptions, we establish existence of global solutions and derive Lyapunov-based convergence guarantees. For convex costs, we prove a sublinear decay in a geometry induced by some reference function, and under a generalized gradient-dominance condition we obtain exponential convergence. We further uncover a duality connection with mirror descent, and use it to establish that the flow of interest solves an infinite-horizon optimal-control problem of which the value function is the Bregman divergence generated by the cost. These results clarify the structure and optimization behavior of nonlinearly preconditioned gradient flows and connect them to known continuous-time models in non-Euclidean optimization.

math.OC↗

arXiv2025

Scaled relative graphs for pairs of operators beyond classical monotonicity

We introduce a generalization of the scaled relative graph (SRG) to pairs of operators, enabling the visualization of their relative incremental properties. This novel SRG framework provides the geometric counterpart for the study of nonlinear resolvents based on paired monotonicity conditions. We demonstrate that these conditions apply to linear operators composed with monotone mappings, a class that notably includes NPN transistors, allowing us to compute the response of multivalued, nonsmooth, and highly nonmonotone electrical circuits.

math.OC↗

arXiv2025

Rethinking PCA Through Duality

Motivated by the recently shown connection between self-attention and (kernel) principal component analysis (PCA), we revisit the fundamentals of PCA. Using the difference-of-convex (DC) framework, we present several novel formulations and provide new theoretical insights. In particular, we show the kernelizability and out-of-sample applicability for a PCA-like family of problems. Moreover, we uncover that simultaneous iteration, which is connected to the classical QR algorithm, is an instance of the difference-of-convex algorithm (DCA), offering an optimization perspective on this longstanding method. Further, we describe new algorithms for PCA and empirically compare them with state-of-the-art methods. Lastly, we introduce a kernelizable dual formulation for a robust variant of PCA that minimizes the $l_1$ deviation of the reconstruction errors.

cs.LG↗

arXiv2025

Nonlinearly Preconditioned Gradient Methods: Momentum and Stochastic Analysis

We study nonlinearly preconditioned gradient methods for smooth nonconvex optimization problems, focusing on sigmoid preconditioners that inherently perform a form of gradient clipping akin to the widely used gradient clipping technique. Building upon this idea, we introduce a novel heavy ball-type algorithm and provide convergence guarantees under a generalized smoothness condition that is less restrictive than traditional Lipschitz smoothness, thus covering a broader class of functions. Additionally, we develop a stochastic variant of the base method and study its convergence properties under different noise assumptions. We compare the proposed algorithms with baseline methods on diverse tasks from machine learning including neural network training.

math.OC↗

arXiv2025

Nonlinearly Preconditioned Gradient Methods under Generalized Smoothness

We analyze nonlinearly preconditioned gradient methods for solving smooth minimization problems. We introduce a generalized smoothness property, based on the notion of abstract convexity, that is broader than Lipschitz smoothness and provide sufficient first- and second-order conditions. Notably, our framework encapsulates algorithms associated with the gradient clipping method and brings out novel insights for the class of $(L_0,L_1)$-smooth functions that has received widespread interest recently, thus allowing us to extend beyond already established methods. We investigate the convergence of the proposed method in both the convex and nonconvex setting.

math.OC↗

arXiv2024

Scaled Relative Graphs for Nonmonotone Operators with Applications in Circuit Theory

The scaled relative graph (SRG) is a powerful graphical tool for analyzing the properties of operators, by mapping their graph onto the complex plane. In this work, we study the SRG of two classes of nonmonotone operators, namely the general class of semimonotone operators and a class of angle-bounded operators. In particular, we provide an analytical description of the SRG of these classes and show that membership of an operator to these classes can be verified through geometric containment of its SRG. To illustrate the importance of these results, we provide several examples in the context of electrical circuits. Most notably, we show that the Ebers-Moll transistor belongs to the class of angle-bounded operators and use this result to compute the response of a common-emitter amplifier using Chambolle-Pock, despite the underlying nonsmoothness and multi-valuedness, leveraging recent convergence results for this algorithm in the nonmonotone setting.

math.OC↗

arXiv2021

A phase defect framework for the analysis of cardiac arrhythmia patterns

During cardiac arrhythmias, dynamical patterns of electrical activation form and evolve, which are of interest to understand and cure heart rhythm disorders. The analysis of these patterns is commonly performed by calculating the local activation phase and searching for phase singularities (PSs), i.e. points around which all phases are present. Here we propose an alternative framework, which focuses on phase defect lines (PDLs) and surfaces (PDSs) as more general mechanisms, which include PSs as a specific case. The proposed framework enables two conceptual unifications: between the local activation time and phase description, and between conduction block lines and the central regions of linear-core rotors. A simple PDL detection method is proposed and applied to data from simulations and optical mapping experiments. Our analysis of ventricular tachycardia in rabbit hearts $(n=6)$ shows that nearly all detected PSs were found on PDLs, but the PDLs had a significantly longer lifespan than the detected PSs. Since the proposed framework revisits basic building blocks of cardiac activation patterns, it can become a useful tool for further theory development and experimental analysis.

nlin.AO↗