Fully Deterministic Shallow Neural Networks with Localized Activations. Part I: Construction and Optimal Sobolev Approximation

Abstract

We construct shallow neural networks with fixed Gaussian and tanh-difference activations on bounded Lipschitz domains $Ω\subset\mathbb{R}^d$, $d\ge2$. For each $s>0$, explicit deterministic rules prescribe all hidden parameters independently of the target $u\in H^s(Ω)$. At frequency resolution $M$, the $N_M\asymp M^d$ prescribed features span a linear space $V_M$. For every sufficiently large integer $N$, we choose $M\asymp N^{1/d}$ with $N_M\le N$ and construct a bounded linear operator $A_N:H^s(Ω)\to V_M$ such that \[ \|u-A_Nu\|_{H^m(Ω)} \le C N^{-(s-m)/d}\|u\|_{H^s(Ω)} \] for every $u\in H^s(Ω)$ and every integer $0\le m<s$, with $C$ independent of $u$, $N$, and $m$. Sobolev width lower bounds show that this rate is optimal among linear spaces of dimension at most $N$ chosen independently of the target. We use explicit spherical cubature nodes for the directions and Gauss-Legendre nodes or equispaced midpoints for the offsets. We show that the exterior offset margin can shrink to zero as $M\to\infty$ while preserving the approximation rate. A margin beyond $\sup_{x\inΩ}|x|$ of order $M^{-1}\log M$ for the Gaussian and $M^{-1}(\log M)^2$ for the tanh difference suffices to truncate the ridge integral and remove exterior midpoint nodes. Numerical experiments in two and three dimensions examine $L^2$ and $H^1$ approximation.