arXiv subjects

Anne Reinarz

Publications and source records attributed to Anne Reinarz.

15 recordsLinked to original sources

arXiv2025

A Performance Analysis of Task Scheduling for UQ Workflows on HPC Systems

Uncertainty Quantification (UQ) workloads are becoming increasingly common in science and engineering. They involve the submission of thousands or even millions of similar tasks with potentially unpredictable runtimes, where the total number is usually not known a priori. A static one-size-fits-all batch script would likely lead to suboptimal scheduling, and native schedulers installed on High Performance Computing (HPC) systems such as SLURM often struggle to efficiently handle such workloads. In this paper, we introduce a new load balancing approach suitable for UQ workflows. To demonstrate its efficiency in a real-world setting, we focus on the GS2 gyrokinetic plasma turbulence simulator. Individual simulations can be computationally demanding, with runtimes varying significantly-from minutes to hours-depending on the high-dimensional input parameters. Our approach uses UQ and Modelling Bridge, which offers a language-agnostic interface to a simulation model, combined with HyperQueue which works alongside the native scheduler. In particular, deploying this framework on HPC systems does not require system-level changes. We benchmark our proposed framework against a standalone SLURM approach using GS2 and a Gaussian Process surrogate thereof. Our results demonstrate a reduction in scheduling overhead by up to three orders of magnitude and a maximum reduction of 38% in CPU time for long-running simulations compared to the naive SLURM approach, while making no assumptions about the job submission patterns inherent to UQ workflows.

cs.DC↗

arXiv2024

Democratizing Uncertainty Quantification

Uncertainty Quantification (UQ) is vital to safety-critical model-based analyses, but the widespread adoption of sophisticated UQ methods is limited by technical complexity. In this paper, we introduce UM-Bridge (the UQ and Modeling Bridge), a high-level abstraction and software protocol that facilitates universal interoperability of UQ software with simulation codes. It breaks down the technical complexity of advanced UQ applications and enables separation of concerns between experts. UM-Bridge democratizes UQ by allowing effective interdisciplinary collaboration, accelerating the development of advanced UQ methods, and making it easy to perform UQ analyses from prototype to High Performance Computing (HPC) scale. In addition, we present a library of ready-to-run UQ benchmark problems, all easily accessible through UM-Bridge. These benchmarks support UQ methodology research, enabling reproducible performance comparisons. We demonstrate UM-Bridge with several scientific applications, harnessing HPC resources even using UQ codes not designed with HPC support.

cs.MS↗

arXiv2023

Lowering the Entry Bar to HPC-Scale Uncertainty Quantification

Treating uncertainties in models is essential in many fields of science and engineering. Uncertainty quantification (UQ) on complex and computationally costly numerical models necessitates a combination of efficient model solvers, advanced UQ methods and HPC-scale resources. The resulting technical complexities as well as lack of separation of concerns between UQ and model experts is holding back many interesting UQ applications. The aim of this paper is to close the gap between advanced UQ methods and advanced models by removing the hurdle of complex software stack integration, which in turn will offer a straightforward way to scale even prototype-grade UQ applications to high-performance resources. We achieve this goal by introducing a parallel software architecture based on UM-Bridge, a universal interface for linking UQ and models. We present three realistic applications from different areas of science and engineering, scaling from single machines to large clusters on the Google Cloud Platform.

cs.DC↗

arXiv2021

Doubt and Redundancy Kill Soft Errors -- Towards Detection and Correction of Silent Data Corruption in Task-based Numerical Software

Resilient algorithms in high-performance computing are subject to rigorous non-functional constraints. Resiliency must not increase the runtime, memory footprint or I/O demands too significantly. We propose a task-based soft error detection scheme that relies on error criteria per task outcome. They formalise how ``dubious'' an outcome is, i.e. how likely it contains an error. Our whole simulation is replicated once, forming two teams of MPI ranks that share their task results. Thus, ideally each team handles only around half of the workload. If a task yields large error criteria values, i.e.~is dubious, we compute the task redundantly and compare the outcomes. Whenever they disagree, the task result with a lower error likeliness is accepted. We obtain a self-healing, resilient algorithm which can compensate silent floating-point errors without a significant performance, I/O or memory footprint penalty. Case studies however suggest that a careful, domain-specific tailoring of the error criteria remains essential.

cs.SE↗

arXiv2021

High Performance Uncertainty Quantification with Parallelized Multilevel Markov Chain Monte Carlo

Numerical models of complex real-world phenomena often necessitate High Performance Computing (HPC). Uncertainties increase problem dimensionality further and pose even greater challenges. We present a parallelization strategy for multilevel Markov chain Monte Carlo, a state-of-the-art, algorithmically scalable Uncertainty Quantification (UQ) algorithm for Bayesian inverse problems, and a new software framework allowing for large-scale parallelism across forward model evaluations and the UQ algorithms themselves. The main scalability challenge presents itself in the form of strong data dependencies introduced by the MLMCMC method, prohibiting trivial parallelization. Our software is released as part of the modular and open-source MIT UQ Library (MUQ), and can easily be coupled with arbitrary user codes. We demonstrate it using the DUNE and the ExaHyPE Engine. The latter provides a realistic, large-scale tsunami model in which identify the source of a tsunami from buoy-elevation data.

cs.MS↗

arXiv2020

Vectorization and Minimization of Memory Footprint for Linear High-Order Discontinuous Galerkin Schemes

We present a sequence of optimizations to the performance-critical compute kernels of the high-order discontinuous Galerkin solver of the hyperbolic PDE engine ExaHyPE -- successively tackling bottlenecks due to SIMD operations, cache hierarchies and restrictions in the software design. Starting from a generic scalar implementation of the numerical scheme, our first optimized variant applies state-of-the-art optimization techniques by vectorizing loops, improving the data layout and using Loop-over-GEMM to perform tensor contractions via highly optimized matrix multiplication functions provided by the LIBXSMM library. We show that memory stalls due to a memory footprint exceeding our L2 cache size hindered the vectorization gains. We therefore introduce a new kernel that applies a sum factorization approach to reduce the kernel's memory footprint and improve its cache locality. With the L2 cache bottleneck removed, we were able to exploit additional vectorization opportunities, by introducing a hybrid Array-of-Structure-of-Array data layout that solves the data layout conflict between matrix multiplications kernels and the point-wise functions to implement PDE-specific terms. With this last kernel, evaluated in a benchmark simulation at high polynomial order, only 2\% of the floating point operations are still performed using scalar instructions and 22.5\% of the available performance is achieved.

cs.MS↗

arXiv2019

Role-Oriented Code Generation in an Engine for Solving Hyperbolic PDE Systems

The development of a high performance PDE solver requires the combined expertise of interdisciplinary teams with respect to application domain, numerical scheme and low-level optimization. In this paper, we present how the ExaHyPE engine facilitates the collaboration of such teams by isolating three roles: application, algorithms, and optimization expert. We thus support team members in letting them focus on their own area of expertise while integrating their contributions into an HPC production code. Inspired by web application development practices, ExaHyPE relies on two custom code generation modules, the Toolkit and the Kernel Generator, which follow a Model-View-Controller architectural pattern on top of the Jinja2 template engine library. Using Jinja2's templates to abstract the critical components of the engine and generated glue code, we isolate the application development from the engine. The template language also allows us to define and use custom template macros that isolate low-level optimizations from the numerical scheme described in the templates. We present three use cases, each focusing on one of our user roles, showcasing how the design of the code generation modules allows to easily expand the solver schemes to support novel demands from applications, to add optimized algorithmic schemes (with reduced memory footprint, e.g.), or provide improved low-level SIMD vectorization support.

cs.MS↗

arXiv2019

On GLM curl cleaning for a first order reduction of the CCZ4 formulation of the Einstein field equations

In this paper we propose an extension of the generalized Lagrangian multiplier method (GLM) of Munz et al. (JCP 2000, JCP 2002), which was originally conceived for the numerical solution of the Maxwell and MHD equations with divergence-type involutions, to the case of hyperbolic PDE systems with curl-type involutions. The key idea here is to solve an augmented PDE system, in which curl errors propagate away via a Maxwell-type evolution system. The new approach is first presented on a simple model problem, in order to explain the basic ideas. Subsequently, we apply it to a strongly hyperbolic first order reduction of the CCZ4 formulation (FO-CCZ4) of the Einstein field equations of general relativity, which is endowed with 11 curl constraints. Several numerical examples, including the long-time evolution of a stable neutron star in anti-Cowling approximation, are presented in order to show the obtained improvements with respect to the standard formulation without special treatment of the curl involution constraints. The main advantages of the proposed GLM approach are its complete independence of the underlying numerical scheme and grid topology and its easy implementation into existing computer codes. However, this flexibility comes at the price of needing to add for each curl involution one additional 3 vector plus another scalar in the augmented system for homogeneous curl constraints, and even two additional scalars for non-homogeneous curl involutions. For the FO-CCZ4 system with 11 homogeneous curl involutions, this means that additional 44 evolution quantities need to be added.

math.NA↗

arXiv2019

A High-Performance Implementation of a Robust Preconditioner for Heterogeneous Problems

We present an efficient implementation of the highly robust and scalable GenEO preconditioner in the high-performance PDE framework DUNE. The GenEO coarse space is constructed by combining low energy solutions of a local generalised eigenproblem using a partition of unity. In this paper we demonstrate both weak and strong scaling for the GenEO solver on over 15,000 cores by solving an industrially motivated problem with over 200 million degrees of freedom. Further, we show that for highly complex parameter distributions arising in certain real-world applications, established methods become intractable while GenEO remains fully effective. The purpose of this paper is two-fold: to demonstrate the robustness and high parallel efficiency of the solver and to document the technical details that are crucial to the efficiency of the code.

cs.MS↗

arXiv2019

ExaHyPE: An Engine for Parallel Dynamically Adaptive Simulations of Wave Problems

ExaHyPE ("An Exascale Hyperbolic PDE Engine") is a software engine for solving systems of first-order hyperbolic partial differential equations (PDEs). Hyperbolic PDEs are typically derived from the conservation laws of physics and are useful in a wide range of application areas. Applications powered by ExaHyPE can be run on a student's laptop, but are also able to exploit thousands of processor cores on state-of-the-art supercomputers. The engine is able to dynamically increase the accuracy of the simulation using adaptive mesh refinement where required. Due to the robustness and shock capturing abilities of ExaHyPE's numerical methods, users of the engine can simulate linear and non-linear hyperbolic PDEs with very high accuracy. Users can tailor the engine to their particular PDE by specifying evolved quantities, fluxes, and source terms. A complete simulation code for a new hyperbolic PDE can often be realised within a few hours - a task that, traditionally, can take weeks, months, often years for researchers starting from scratch. In this paper, we showcase ExaHyPE's workflow and capabilities through real-world scenarios from our two main application areas: seismology and astrophysics.

cs.MS↗

arXiv2019

High-performance dune modules for solving large-scale, strongly anisotropic elliptic problems with applications to aerospace composites

The key innovation in this paper is an open-source, high-performance iterative solver for high contrast, strongly anisotropic elliptic partial differential equations implemented within dune-pdelab. The iterative solver exploits a robust, scalable two-level additive Schwarz preconditioner, GenEO (Spillane et al. 2014). The development of this solver has been motivated by the need to overcome the limitations of commercially available modeling tools for solving structural analysis simulations in aerospace composite applications. Our software toolbox dune-composites encapsulates the mathematical complexities of the underlying packages within an efficient C++ framework, providing an application interface to our new high-performance solver. We illustrate its use on a range of industrially motivated examples, which should enable other scientists to build on and extend dune-composites and the GenEO preconditioner for use in their own applications. We demonstrate the scalability of the solver on more than 15,000 cores of the UK national supercomputer Archer, solving an aerospace composite problem with over 200 million degrees of freedom in a few minutes. This scale of computation brings composites problems that would otherwise be unthinkable into the feasible range. To demonstrate the wider applicability of the new solver, we also confirm the robustness and scalability of the solver on SPE10, a challenging benchmark in subsurface flow/reservoir simulation.

math.NA↗

arXiv2018

Influence of A-Posteriori Subcell Limiting on Fault Frequency in Higher-Order DG Schemes

Soft error rates are increasing as modern architectures require increasingly small features at low voltages. Due to the large number of components used in HPC architectures, these are particularly vulnerable to soft errors. Hence, when designing applications that run for long time periods on large machines, algorithmic resilience must be taken into account. In this paper we analyse the inherent resiliency of a-posteriori limiting procedures in the context of the explicit ADER DG hyperbolic PDE solver ExaHyPE. The a-posteriori limiter checks element-local high-order DG solutions for physical admissibility, and can thus be expected to also detect hardware-induced errors. Algorithmically, it can be interpreted as element-local checkpointing and restarting of the solver with a more robust finite volume scheme on a fine subgrid. We show that the limiter indeed increases the resilience of the DG algorithm, detecting and correcting particularly those faults which would otherwise lead to a fatal failure.

physics.comp-ph↗

arXiv2018

Sparse Grid Approximation Spaces for Space-Time Boundary Integral Formulations of the Heat Equation

The aim of this paper is to develop stable and accurate numerical schemes for boundary integral formulations of the heat equation with Dirichlet boundary conditions. The accuracy of Galerkin discretisations for the resulting boundary integral formulations depends mainly on the choice of discretisation space. We develop a-priori error analysis utilising a proof technique that involves norm equivalences in hierarchical wavelet subspace decompositions. We apply this to a full tensor product discretisation, showing improvements over existing results, particularly for discretisation spaces having low polynomial degrees. We then use the norm equivalences to show that an anisotropic sparse grid discretisation yields even higher convergence rates. Finally, a simple adaptive scheme is proposed to suggest an optimal shape for the sparse grid index sets.

math.NA↗

arXiv2018

A Bayesian Framework for Assessing the Strength Distribution of Composite Structures with Random Defects

This paper presents a novel stochastic framework to quantify the knock down in strength from out-of-plane wrinkles at the coupon level. The key innovation is a Markov Chain Monte Carlo algorithm which rigorously derives the stochastic distribution of wrinkle defects directly informed from image data of defects. The approach significantly reduces uncertainty in the parameterization of stochastic numerical studies on the effects of defects. To demonstrate our methodology, we present an original stochastic study to determine the distribution of strength of corner bend samples with random out-plane wrinkle defects. The defects are parameterized by stochastic random fields defined using Karhunen-Lo\'{e}ve (KL) modes. The distribution of KL coefficients are inferred from misalignment data extracted from B-Scan data using a modified version of Multiple Field Image Analysis. The strength distribution is estimated, by embedding wrinkles into high fidelity FE simulations using the high performance toolbox 'dune-composites' from which we observe severe knockdowns of $74\%$ with a probability of $1/200$. Supported by the literature our results highlight the strong correlation between maximum misalignment and knockdown in coupon strength. This observations allows us to define a surrogate model providing fast assessment of predicted strength informed from stochastic simulations utilizing both observed wrinkle data and high fidelity finite element models.

stat.AP↗

arXiv2017

dune-composites -- A New Framework for High-Performance Finite Element Modelling of Laminates

Finite element (FE) analysis has the potential to offset much of the expensive experimental testing currently required to certify aerospace laminates. However, large numbers of degrees of freedom are necessary to model entire aircraft components whilst accurately resolving micro-scale defects. The new module dune-composites, implemented within DUNE by the authors, provides a tool to efficiently solve large-scale problems using novel iterative solvers. The key innovation is a preconditioner that guarantees a constant number of iterations regardless of the problem size. Its robustness has been shown rigorously in Spillane et al. (Numer. Math. 126, 2014) for isotropic problems. For anisotropic problems in composites it is verified numerically for the first time in this paper. The parallel implementation in DUNE scales almost optimally over thousands of cores. To demonstrate this, we present an original numerical study, varying the shape of a localised wrinkle and the effect this has on the strength of a curved laminate. This requires a high-fidelity mesh containing at least four layers of quadratic elements across each ply and interface layer, underlining the need for dune-composites, which can achieve run times of just over 2 minutes on 2048 cores for realistic composites problems with 173 million degrees of freedom.

math.NA↗