Adaptive Randomized Neural Networks with Locally Activation Function: Theory and Algorithm for Solving PDEs

Abstract

This paper establishes an approximation theory and develop an adaptive computational framework for randomized neural networks (RaNNs). For RaNNs of the form $\sum_{i=1}^{N} W_i \sigma(A_i\cdot x+B_i)$ with hidden parameters uniformly sampled from a bounded set of scale $M$, we show that the choice $M \asymp N^{p/[2((p-1)(d+1)+p\eta)]}$ yields an expected $W^{k,p}$-approximation error of order $\mathcal{O}\left(N^{-p\eta/[2((p-1)(d+1)+p\eta)]}\right)$, where $\eta$ is associated with the regularity of the target function in a generalized Barron spectral space. This theoretical result demonstrates that lower regularity requires a larger sampling range for the hidden parameters. Motivated by the relationship between $M$ and $\eta$, we introduce an adaptive physics-informed RaNN (PIRaNN) method that which couples parameter sampling with an adaptive partition of unity via local affine scaling. Residual-based a posteriori error indicators and D\"{o}rfler marking are used to drive local refinement. Numerical experiments, including the 1D viscous Burgers' equation and 2D/3D problems with localized peaks and L-shaped corner singularities, demonstrate that our method preferentially refines regions exhibiting large gradients and singularities. Compared to standard non-adaptive RaNNs, the adaptive PIRaNN effectively captures complex local features with significantly enhanced accuracy and efficiency.