arXiv · 2502.05074
Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
Abstract
We derive a novel deterministic equivalence for the two-point function of a random matrix resolvent. Using this result, we give a unified derivation of the performance of a wide variety of high-dimensional linear models trained with stochastic gradient descent. This includes high-dimensional linear regression, kernel regression, and linear random feature models. Our results include previously known asymptotics as well as novel ones.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Alexander Atanasov, Blake Bordelon, Jacob A. Zavatone-Veth, Courtney Paquette, Cengiz Pehlevan. 2025-02-07. Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models. https://doi.org/10.4310/atmp.260412235902
Cite the original work for its findings. Save a collection to share your selection of sources.