arXiv subjects

Alexander Heinlein

Publications and source records attributed to Alexander Heinlein.

At least 19 recordsLinked to original sources

arXiv2026

Model reduction of port-Hamiltonian systems via neural networks

In this paper, we consider structure-preserving model reduction of port-Hamiltonian (pH) systems which extend classical Hamiltonian systems with dissipation and an input-output port. These pH systems are often used in multi-physics systems, as the interconnection of one or more \pH systems results again in a pH system. If particularly the system matrices associated with the interconnection and/or dissipation of a pH system are state-dependent, then the evaluation of standard reduced-order models (ROMs) may depend on the dimension of the original full-order model, resulting in high computational costs. To circumvent these high costs, we propose to use structure-preserving neural networks. In particular, we perform two steps: (1) we use the generalized manifold Galerkin projection to project the pH system onto the reduced space; then (2) we train a neural network to learn the map from the reduced-order state to the reduced-order interconnection and dissipation system matrices. To ensure that the resulting ROM is again a pH system, the architecture of the neural network is chosen such that the skew-symmetry and positive semi-definiteness of the reduced-order systems matrices are maintained. In a numerical example, we consider a nonlinear mass-spring-damper system with state-dependent system matrices. The numerical results show that the proposed method achieves a significant computational speed-up compared to the original \ROM with comparable accuracy.

math.NA↗

arXiv2026

Two-level domain-decomposition AdaGrad method for scalable training of graph neural networks

Graph neural networks (GNNs) have emerged as a powerful framework for learning from graph-structured data. However, their efficient training remains challenging, particularly in distributed computing environments. This challenge arises from the use of message passing, which couples all graph nodes, leading to expensive optimization steps, high memory requirements, and substantial communication overhead. To alleviate these limitations, we propose a novel domain-decomposition (DD) variant of AG2m, an AdaGrad method enhanced with second-order curvature information and momentum, denoted by DD-AG2m. The proposed DD-AG2m alternates between AG2m optimization on the original (global) graph and AG2m optimization on the partitioned graphs. To incorporate global information at reduced cost, we further introduce a two-level variant (2DD-AG2m) that performs global optimization steps on a coarse graph obtained by randomly subsampling nodes within each subdomain. Numerical experiments spanning graph classification, node-level regression, and spatiotemporal forecasting tasks demonstrate that the proposed DD methods reduce the computational cost required to achieve the same predictive performance by a factor of 4-8. Moreover, for the fixed computational cost, they improve the predictive performance of GNNs by up to 22% compared with the baseline AG2m.

math.NA↗

arXiv2026

Flow-based conditional cardiac anatomy generation for virtual cohorts

Cardiac digital twin research is moving from subject-specific anatomical replicas toward virtual cohorts that represent clinically relevant population subgroups. Yet access to representative imaging-derived anatomy datasets remains limited by cohort size, subgroup sparsity, and data-sharing constraints. Conditional generative models could help address this gap, but virtual cohorts are useful only if they preserve realistic, metadata-dependent anatomical variability. Existing cardiac anatomy generators largely rely on conditional variational autoencoders (cVAEs), which couple representation learning and metadata conditioning through a shared regularized latent prior. We introduce CAN-FLOW, a two-step Conditional ANatomy generation framework based on normalizing FLOWs that first learns geometry-only latent representations of diffeomorphic cardiac shape momenta and then models their sex-, age-, and body-mass-index-dependent distribution with a conditional normalizing flow. We trained CAN-FLOW on 2,208 healthy UK Biobank subjects and compared it with cVAEs across regularization strengths. CAN-FLOW generated plausible stochastic biventricular anatomies that better reproduced clinical phenotype distributions, metadata-dependent trends, subgroup variability, point-cloud coverage, and high-dimensional shape variability. Together, these results establish CAN-FLOW as a shareable framework for generating realistic, stochastically varying, metadata-conditioned biventricular anatomies for virtual cohort construction and in silico clinical trial workflows.

cs.LG↗

arXiv2026

The Error of Deep Operator Networks Is the Sum of Its Parts: Branch-Trunk and Mode Error Decompositions

Operator learning has the potential to strongly impact scientific computing by learning solution operators for differential equations, potentially accelerating multi-query tasks such as design optimization and uncertainty quantification by orders of magnitude. Despite proven universal approximation properties, deep operator networks (DeepONets) often exhibit limited accuracy and generalization in practice, which hinders their adoption. Understanding these limitations is therefore crucial for further advancing the approach. This work analyzes performance limitations of the classical DeepONet architecture. It is shown that the approximation error is dominated by the branch network when the internal dimension is sufficiently large, and that the learned trunk basis can often be replaced by classical basis functions without a significant impact on performance. To investigate this further, a modified DeepONet is constructed in which the trunk network is replaced by the left singular vectors of the training solution matrix. This modification yields several key insights. First, for examples involving the KdV and Burgers equations, a spectral bias in the branch network is observed, with coefficients of dominant, low-frequency modes learned more effectively. Second, through the interplay of the singular-value weighting and the optimizer's neglect of small modes, the branch error is dominated by modes with large and intermediate singular values. Third, using a shared branch network for all mode coefficients, as in the standard architecture, improves generalization of small modes compared to a stacked architecture in which coefficients are computed separately. Finally, detrimental coupling between modes in parameter space is identified, which weakens with increasing network width.

cs.LG↗

arXiv2026

Are Deep Learning Based Hybrid PDE Solvers Reliable? Why Training Paradigms and Update Strategies Matter

Deep learning-based hybrid iterative methods (DL-HIMs) integrate classical numerical solvers with neural operators, utilizing their complementary spectral biases to accelerate convergence. Despite this promise, many DL-HIMs stagnate at false fixed points where neural updates vanish while the physical residual remains large, raising questions about reliability in scientific computing. In this paper, we provide evidence that performance is highly sensitive to training paradigms and update strategies, even when the neural architecture is fixed. Through a detailed study of a DeepONet-based hybrid iterative numerical transferable solver (HINTS) and an FFT-based Fourier neural solver (FNS), we show that significant physical residuals can persist when training objectives are not aligned with solver dynamics and problem physics. We further examine Anderson acceleration (AA) and demonstrate that its classical form is ill-suited for nonlinear neural operators. To overcome this, we introduce physics-aware Anderson acceleration (PA-AA), which minimizes the physical residual rather than the fixed-point update. Numerical experiments confirm that PA-AA restores reliable convergence in substantially fewer iterations. These findings provide a concrete answer to ongoing controversies surrounding AI-based PDE solvers: reliability hinges not only on architectures but on physically informed training and iteration design.

math.NA↗

arXiv2026

Multi-Preconditioned LBFGS for Training Finite-Basis PINNs

A multi-preconditioned LBFGS (MP-LBFGS) algorithm is introduced for training finite-basis physics-informed neural networks (FBPINNs). The algorithm is motivated by the nonlinear additive Schwarz method and exploits the domain-decomposition-inspired additive architecture of FBPINNs, in which local neural networks are defined on subdomains, thereby localizing the network representation. Parallel, subdomain-local quasi-Newton corrections are then constructed on the corresponding local parts of the architecture. A key feature is a novel nonlinear multi-preconditioning mechanism, in which subdomain corrections are optimally combined through the solution of a low-dimensional subspace minimization problem. Numerical experiments indicate that MP-LBFGS can improve convergence speed, as well as model accuracy over standard LBFGS while incurring lower communication overhead.

math.NA↗

arXiv2025

Two-level additive Schwarz preconditioners for reduced integration methods

Incompressible fluid flow problems appear frequently in different applications. The discretization of such problems may result in large and ill-conditioned systems of linear equations. We consider the case of the Stokes equations discretized using a reduced integration method which approximates the incompressibility constraint by a penalty term thus allowing the problem to be solved only in terms of the velocity unknowns. We investigate the numerical scalability of a two-level overlapping additive Schwarz method with a reduced dimension generalized Dryja-Smith-Widlund (RGDSW) coarse space. In addition, we discuss the parallel implementation of the examples using the Fast and Robust Overlapping Schwarz (FROSch) package for additive Schwarz preconditioners and the NGSolve library, which implements multiple finite element space formulations.

math.NA↗

arXiv2025

Domain-Decomposed Graph Neural Network Surrogate Modeling for Ice Sheets

Accurate yet efficient surrogate models are essential for large-scale simulations of partial differential equations (PDEs), particularly for uncertainty quantification (UQ) tasks that demand hundreds or thousands of evaluations. We develop a physics-inspired graph neural network (GNN) surrogate that operates directly on unstructured meshes and leverages the flexibility of graph attention. To improve both training efficiency and generalization properties of the model, we introduce a domain decomposition (DD) strategy that partitions the mesh into subdomains, trains local GNN surrogates in parallel, and aggregates their predictions. We then employ transfer learning to fine-tune models across subdomains, accelerating training and improving accuracy in data-limited settings. Applied to ice sheet simulations, our approach accurately predicts full-field velocities on high-resolution meshes, substantially reduces training time relative to training a single global surrogate model, and provides a ripe foundation for UQ objectives. Our results demonstrate that graph-based DD, combined with transfer learning, provides a scalable and reliable pathway for training GNN surrogates on massive PDE-governed systems, with broad potential for application beyond ice sheet dynamics.

cs.LG↗

arXiv2025

Sharpened PCG Iteration Bound for High-Contrast Heterogeneous Scalar Elliptic PDEs

A new iteration bound for the preconditioned conjugate gradient (PCG) method is presented that more accurately captures convergence for systems with clustered eigenspectra, where the classical condition number-based bound is too pessimistic. By using the edge eigenvalues of each cluster in the spectral distribution, the bound is shown to be orders of magnitude sharper than the classical bound for certain examples. Its effectiveness is demonstrated on a high-contrast elliptic PDE preconditioned with a two-level overlapping Schwarz preconditioner, where the performance of different (algebraic) coarse spaces is successfully distinguished. A key contribution of this work is the observation that, for certain high-contrast problems, simpler coarse spaces can be made competitive in terms of PCG convergence. Conversely, more complex preconditioners are not always required. Finally, it is shown that the bound can be estimated effectively from Ritz values computed during early PCG iterations.

math.NA↗

arXiv2025

Domain decomposition architectures and Gauss-Newton training for physics-informed neural networks

Approximating the solutions of boundary value problems governed by partial differential equations with neural networks is challenging, largely due to the difficult training process. This difficulty can be partly explained by the spectral bias, that is, the slower convergence of high-frequency components, and can be mitigated by localizing neural networks via (overlapping) domain decomposition. We combine this localization with the Gauss-Newton method as the optimizer to obtain faster convergence than gradient-based schemes such as Adam; this comes at the cost of solving an ill-conditioned linear system in each iteration. Domain decomposition induces a block-sparse structure in the otherwise dense Gauss-Newton system, reducing the computational cost per iteration. Our numerical results indicate that combining localization and Gauss-Newton optimization is promising for neural network-based solvers for partial differential equations.

math.NA↗

arXiv2025

Resolving Extreme Data Scarcity by Explicit Physics Integration: An Application to Groundwater Heat Transport

Real-world flow applications in complex scientific and engineering domains, such as geosciences, challenge classical simulation methods due to large spatial domains, high spatio-temporal resolution requirements, and potentially strong material heterogeneities that lead to ill-conditioning and long runtimes. While machine learning-based surrogate models can reduce computational cost, they typically rely on large training datasets that are often unavailable in practice. To address data-scarce settings, we revisit the structure of advection-diffusion problems and decompose them into multiscale processes of locally and globally dominated components, separating spatially localized interactions and long-range effects. We propose a Local-Global Convolutional Neural Network (LGCNN) that combines a lightweight numerical model for global transport with two convolutional neural networks addressing processes of a more local nature. We demonstrate the performance of our method on city-scale geothermal heat pump interaction modeling and show that, even when trained on fewer than five simulations, LGCNN generalizes to arbitrarily larger domains, and can be successfully transferred to real subsurface parameter maps from the Munich region, Germany.

cs.LG↗

arXiv2025

Monolithic and Block Overlapping Schwarz Preconditioners for the Incompressible Navier-Stokes Equations

Monolithic preconditioners applied to the linear systems arising during the solution of the discretized incompressible Navier-Stokes equations are typically more robust than preconditioners based on incomplete block factorizations. Lower number of iterations and a reduced sensitivity to parameters like velocity and viscosity can significantly outweigh the additional cost for their setup. Different monolithic preconditioning techniques are introduced and compared to a selection of block preconditioners. In particular, two-level additive overlapping Schwarz methods (OSM) are used to set up monolithic preconditioners and to approximate the inverses arising in the block preconditioners. GDSW-type (Generalized Dryja-Smith-Widlund) coarse spaces are used for the second level. These highly scalable, parallel preconditioners have been implemented in the solver framework FROSch (Fast and Robust Overlapping Schwarz), which is part of the software library Trilinos. The new GDSW-type coarse space GDSW* is introduced; combining it with other techniques results in a robust algorithm. The block preconditioners PCD (Pressure Convection-Diffusion), SIMPLE (Semi-Implicit Method for Pressure Linked Equations), and LSC (Least-Squares Commutator) are considered to various degrees. The OSM for the monolithic as well as the block approach allows the optimized combination of different coarse spaces for the velocity and pressure components, enabling the use of tailored coarse spaces. The numerical and parallel performance of the different preconditioning methods for finite element discretizations of stationary as well as time-dependent incompressible fluid flow problems is investigated and compared. Their robustness is analyzed for a range of Reynolds and Courant-Friedrichs-Lewy (CFL) numbers with respect to a realistic problem setting.

math.NA↗

arXiv2025

Conformalized-KANs: Uncertainty Quantification with Coverage Guarantees for Kolmogorov-Arnold Networks (KANs) in Scientific Machine Learning

This paper explores uncertainty quantification (UQ) methods in the context of Kolmogorov-Arnold Networks (KANs). We apply an ensemble approach to KANs to obtain a heuristic measure of UQ, enhancing interpretability and robustness in modeling complex functions. Building on this, we introduce Conformalized-KANs, which integrate conformal prediction, a distribution-free UQ technique, with KAN ensembles to generate calibrated prediction intervals with guaranteed coverage. Extensive numerical experiments are conducted to evaluate the effectiveness of these methods, focusing particularly on the robustness and accuracy of the prediction intervals under various hyperparameter settings. We show that the conformal KAN predictions can be applied to recent extensions of KANs, including Finite Basis KANs (FBKANs) and multifideilty KANs (MFKANs). The results demonstrate the potential of our approaches to improve the reliability and applicability of KANs in scientific machine learning.

cs.LG↗

arXiv2025

Numerical study on hyper parameter settings for neural network approximation to partial differential equations

Approximate solutions of partial differential equations (PDEs) obtained by neural networks are highly affected by hyper parameter settings. For instance, the model training strongly depends on loss function design, including the choice of weight factors for different terms in the loss function, and the sampling set related to numerical integration; other hyper parameters, like the network architecture and the optimizer settings, also impact the model performance. On the other hand, suitable hyper parameter settings are known to be different for different model problems and currently no universal rule for the choice of hyper parameters is known. In this paper, for second order elliptic model problems, various hyper parameter settings are tested numerically to provide a practical guide for efficient and accurate neural network approximation. While a full study of all possible hyper parameter settings is not possible, we focus on studying the formulation of the PDE loss as well as the incorporation of the boundary conditions, the choice of collocation points associated with numerical integration schemes, and various approaches for dealing with loss imbalances will be extensively studied on various model problems; in addition to various Poisson model problems, also a nonlinear and an eigenvalue problem are considered.

math.NA↗

arXiv2025

Trilinos: Enabling Scientific Computing Across Diverse Hardware Architectures at Scale

Trilinos is a community-developed, open-source software framework that facilitates building large-scale, complex, multiscale, multiphysics simulation code bases for scientific and engineering problems. Since the Trilinos framework has undergone substantial changes to support new applications and new hardware architectures, this document is an update to ``An Overview of the Trilinos project'' by Heroux et al. (ACM Transactions on Mathematical Software, 31(3):397-423, 2005). It describes the design of Trilinos, introduces its new organization in product areas, and highlights established and new features available in Trilinos. Particular focus is put on the modernized software stack based on the Kokkos ecosystem to deliver performance portability across heterogeneous hardware architectures. This paper also outlines the organization of the Trilinos community and the contribution model to help onboard interested users and contributors.

cs.MS↗

arXiv2024

Overlapping Schwarz Preconditioners for Randomized Neural Networks with Domain Decomposition

Randomized neural networks (RaNNs), in which hidden layers remain fixed after random initialization, provide an efficient alternative for parameter optimization compared to fully parameterized networks. In this paper, RaNNs are integrated with overlapping Schwarz domain decomposition in two (main) ways: first, to formulate the least-squares problem with localized basis functions, and second, to construct overlapping preconditioners for the resulting linear systems. In particular, neural networks are initialized randomly in each subdomain based on a uniform distribution and linked through a partition of unity, forming a global solution that approximates the solution of the partial differential equation. Boundary conditions are enforced through a constraining operator, eliminating the need for a penalty term to handle them. Principal component analysis (PCA) is employed to reduce the number of basis functions in each subdomain, yielding a linear system with a lower condition number. By constructing additive and restricted additive Schwarz preconditioners, the least-squares problem is solved efficiently using the Conjugate Gradient (CG) and Generalized Minimal Residual (GMRES) methods, respectively. Our numerical results demonstrate that the proposed approach significantly reduces computational time for multi-scale and time-dependent problems. Additionally, a three-dimensional problem is presented to demonstrate the efficiency of using the CG method with an AS preconditioner, compared to an QR decomposition, in solving the least-squares problem.

math.NA↗

arXiv2024

PACMANN: Point Adaptive Collocation Method for Artificial Neural Networks

Physics-Informed Neural Networks (PINNs) have emerged as a tool for approximating the solution of Partial Differential Equations (PDEs) in both forward and inverse problems. PINNs minimize a loss function which includes the PDE residual determined for a set of collocation points. Previous work has shown that the number and distribution of these collocation points have a significant influence on the accuracy of the PINN solution. Therefore, the effective placement of these collocation points is an active area of research. Specifically, available adaptive collocation point sampling methods have been reported to scale poorly in terms of computational cost when applied to high-dimensional problems. In this work, we address this issue and present the Point Adaptive Collocation Method for Artificial Neural Networks (PACMANN). PACMANN incrementally moves collocation points toward regions of higher residuals using gradient-based optimization algorithms guided by the gradient of the PINN loss function, that is, the squared PDE residual. We apply PACMANN for forward and inverse problems, and demonstrate that this method matches the performance of state-of-the-art methods in terms of the accuracy/efficiency tradeoff for the low-dimensional problems, while outperforming available approaches for high-dimensional problems. Key features of the method include its low computational cost and simplicity of integration into existing physics-informed neural network pipelines. The code is available at https://github.com/CoenVisser/PACMANN.

math.NA↗

arXiv2024

Deep operator network models for predicting post-burn contraction

Burn injuries present a significant global health challenge. Among the most severe long-term consequences are contractures, which can lead to functional impairments and disfigurement. Understanding and predicting the evolution of post-burn wounds is essential for developing effective treatment strategies. Traditional mathematical models, while accurate, are often computationally expensive and time-consuming, limiting their practical application. Recent advancements in machine learning, particularly in deep learning, offer promising alternatives for accelerating these predictions. This study explores the use of a deep operator network (DeepONet), a type of neural operator, as a surrogate model for finite element simulations, aimed at predicting post-burn contraction across multiple wound shapes. A DeepONet was trained on three distinct initial wound shapes, with enhancement made to the architecture by incorporating initial wound shape information and applying sine augmentation to enforce boundary conditions. The performance of the trained DeepONet was evaluated on a test set including finite element simulations based on convex combinations of the three basic wound shapes. The model achieved an $R^2$ score of $0.99$, indicating strong predictive accuracy and generalization. Moreover, the model provided reliable predictions over an extended period of up to one year, with speedups of up to 128-fold on CPU and 235-fold on GPU, compared to the numerical model. These findings suggest that DeepONets can effectively serve as a surrogate for traditional finite element methods in simulating post-burn wound evolution, with potential applications in medical treatment planning.

math.NA↗