arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2209.04036

Functional dimension of feedforward ReLU neural networks

J. Elisenda Grigsby·Kathryn Lindsey·Robert Meyerhoff·Chenxi Wu
View original source ↗Read source document ↗Export metadata ↓

Abstract

It is well-known that the parameterized family of functions representable by fully-connected feedforward neural networks with ReLU activation function is precisely the class of piecewise linear functions with finitely many pieces. It is less well-known that for every fixed architecture of ReLU neural network, the parameter space admits positive-dimensional spaces of symmetries, and hence the local functional dimension near any given parameter is lower than the parametric dimension. In this work we carefully define the notion of functional dimension, show that it is inhomogeneous across the parameter space of ReLU neural network functions, and continue an investigation - initiated in [14] and [5] - into when the functional dimension achieves its theoretical maximum. We also study the quotient space and fibers of the realization map from parameter space to function space, supplying examples of fibers that are disconnected, fibers upon which functional dimension is non-constant, and fibers upon which the symmetry group acts non-transitively.

Explore related subjects

math.MGcs.LGmath.GT

Keep this discovery

BibTeXRIS

J. Elisenda Grigsby, Kathryn Lindsey, Robert Meyerhoff, Chenxi Wu. 2022-09-08. Functional dimension of feedforward ReLU neural networks. https://doi.org/10.1016/j.aim.2025.110636

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Tile sets consisting of two types of concave polygons derived from periodic tilings corresponding to non-periodic tilings with hat and turtle tiles

Using a convex pentagonal monotile belonging to the Type 5 family, we investigate the relationships among the hat tile, turtle tile, and Tile$(1, 1)$. By applying Sugimoto's Perspective and Amfirifma's Perspective, we obtain four types of concave polygons, AH-tile, BH-tile, AT-tile, and BT-tile, each having Heesch number 1 under the conditions considered. We show that these polygons correspond to clusters used to generate the non-periodic tilings $\mathscr{T}_h$ and $\mathscr{T}_s$. We further discuss the possibility that tile sets consisting of pairs selected from these polygons may correspond to $\textit{ASPmr}\{\text{A-tile}, \text{B-tile}\}$.

math.MG↗
arXiv2026

The mean distance to a simple closed curve on the sphere

Kimberling's Problem 10 asks for a simple closed curve of prescribed length $L$ (in particular, $L=4\pi$) on the unit sphere minimizing the mean geodesic distance $\mathcal{J}$ from a point of the sphere to the curve. For a positive integer $n$, put $\vartheta_{n}=\pi/(2n)$ and $L_{n}=2\pi/\sin\vartheta_{n}$. We show that the minimum of $\mathcal{J}$ over rectifiable simple closed curves of length at most $L_{n}$ equals $\vartheta_{n}-\tan(\vartheta_{n}/2)$, that it is attained only by curves of length exactly $L_{n}$, and that the sphere-filling ropes $\beta^{n,k}$ of Gerlach and von der Mosel attain it. Kimberling's case is $n=3$: at $L=4\pi$ the minimum is $\pi/6+\sqrt{3}-2=0.255649\ldots$, attained by an explicit six-arc curve and by its mirror image. For $L\le2\pi$ we determine $J(L)$, the infimum of $\mathcal{J}$ over curves of length $L$, exactly: it equals $\pi/2-L/(2\pi)$, attained precisely by the circles of length $L$. At the lengths $L_{n}$ we do not classify all minimizers, but show that every one of them bisects the sphere into two disks of area $2\pi$ and inradius $\vartheta_{n}$ whose inward collars have the largest possible area at every depth. The great circle is the only minimizer for $n=1$, and the $\beta^{n,k}$ are, up to congruence, the only ones of thickness at least $\sin\vartheta_{n}$. For arbitrary $L$ the function $J$ is nonincreasing, and together with the above this brackets it between two explicit values.

math.MG↗
arXiv2026

The topology of Gromov--Hausdorff space

We prove that the Gromov--Hausdorff space is homeomorphic to the Hilbert space. This paper is divided into four parts. In Part I, we construct an assignment of a full-support probability measure to every nonempty compact metric space that respects isometries and is continuous for simultaneous Hausdorff convergence of the spaces and weak convergence of the measures. In Part II, we use these measures to construct finite-dimensional local models whose induced pseudometrics approximate the original distances uniformly and whose norms and point maps vary continuously up to orthogonal changes of coordinates. In Part III, we use the local models to prove that the Gromov--Hausdorff space is an absolute retract for all metrizable spaces. In Part IV, we establish a discrete approximation property and conclude that the space of isometry classes of nonempty compact metric spaces is homeomorphic to the real separable infinite-dimensional Hilbert space.

math.MG↗