arXiv Science⌕ Search

arXiv · 2611.13231

Block-Sparse Gauss-Newton Optimization of Finite Basis Physics-Informed Neural Networks: Sparse Automatic Differentiation Approach

Abstract

In this work, we develop an efficient Gauss-Newton optimization approach for finite basis physics-informed neural networks (FBPINNs). The key ingredient is the overlapping domain decomposition architecture of FBPINNs, which localizes the influence of each subnetwork and induces block sparsity in both the residual Jacobian and the corresponding Gauss-Newton matrix. We characterize this structure and exploit it through sparse automatic differentiation, using graph coloring to partition structurally orthogonal Jacobian columns or rows. This reduces differentiation costs and memory requirements while preserving all structurally nonzero derivative information. The resulting compressed Jacobian representations allow us to assemble the Gauss-Newton system directly in block-sparse form, enabling the use of sparse direct solvers and preconditioned iterative methods. In this way, the otherwise expensive second-order optimization problem is reduced to scalable sparse linear algebra. We demonstrate the approach on multiscale PDEs, where accurate approximation requires rich neural network parametrizations and large numbers of collocation points. Numerical experiments show that our block-sparse Gauss-Newton method attains high accuracy and substantially accelerates optimization relative to first-order baselines while remaining computationally tractable for large problem sizes.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Denis Korolev, Adrian Hill, Jeeeun Lee. 2026-11-09. Block-Sparse Gauss-Newton Optimization of Finite Basis Physics-Informed Neural Networks: Sparse Automatic Differentiation Approach. https://arxiv.science/abs/2611.13231

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Adaptive mesh refinement for the Landau-Lifshitz-Gilbert equation

We propose a new adaptive algorithm for the approximation of the Landau--Lifshitz--Gilbert equation via a higher-order tangent plane scheme. We show that the adaptive approximation satisfies an energy inequality and demonstrate numerically that the adaptive algorithm outperforms uniform approaches.

math.NA↗

A one-dimensional model of catheter aspiration in blood vessels

Aspiration thrombectomy is a primary endovascular treatment option for acute ischemic stroke caused by large vessel occlusions, in which a catheter is navigated to the occlusion site and suction is applied to remove the thrombus. This work introduces a reduced one-dimensional model for aspiration in blood vessels that accounts for the elasticity of both the vessel wall and the catheter. The inclusion of vessel wall visco-elasticity transforms the governing equation for the flow rate into a parabolic form, enabling an accurate resolution of the diffuse pressure gradients near the catheter tip observed in CFD simulations. The numerical treatment is based on a relaxation of the hyperbolic subsystem that facilitates nodal solvers, enabling efficient coupling between catheterized and uncatheterized vessel segments, including bifurcations and the catheter tip. A splitting strategy ensures that the visco-elastic terms incur only negligible additional computational cost relative to the purely hyperbolic model. The model is benchmarked against three-dimensional CFD simulations and reference data from the literature. Numerical experiments investigating the role of catheter elasticity and suction force on the hemodynamics are presented, and an uncertainty quantification study demonstrates the suitability of the framework for efficient parameter studies. The results indicate that the proposed model resolves blood flow comparably accurate to three-dimensional simulations at a fraction of the computational cost. While further in vitro and in vivo validation is required, the framework represents a promising step toward computationally efficient preoperative planning and uncertainty-aware decision support in aspiration thrombectomy.

math.NA↗

Exploiting the nonzero diagonal pattern in matrix function computations

We consider the task of approximating a matrix function $f(A)$, where $A$ is a matrix in which only a relatively small number of (not necessarily consecutive) sub- and superdiagonals contain nonzero entries. Approximating $f$ by a low-degree polynomial $p$ allows us to obtain sparse approximations to $f(A)$, which one can efficiently work with (while, in general, $f(A)$ is a dense matrix, even when $A$ is sparse). Our approach is based on carefully inspecting the locations where nonzeros can occur in $p(A)$, and identifying the entries in $A$ that influence them. In particular, we illustrate how this approach can be used for efficiently approximating the trace of $f(A)$ and identify how this approach is related to established (stochastic) probing methods for trace estimation. Another application area in which our approach works particularly well is the computation of functions of Toeplitz matrices. Here, studying the sparsity pattern of $p(A)$ allows us to reduce the computation of the whole matrix polynomial to that of a single small-scale submatrix, yielding an algorithm that scales exceptionally well to large problem sizes.

math.NA↗