arXiv Science⌕ Search

SEARCH · arXiv Science

Results for “cs.MS”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48Linked to original sources
arXiv2020

Role-Oriented Code Generation in an Engine for Solving Hyperbolic PDE Systems

The development of a high performance PDE solver requires the combined expertise of interdisciplinary teams with respect to application domain, numerical scheme and low-level optimization. In this paper, we present how the ExaHyPE engine facilitates the collaboration of such teams by isolating three roles: application, algorithms, and optimization expert. We thus support team members in letting them focus on their own area of expertise while integrating their contributions into an HPC production code. Inspired by web application development practices, ExaHyPE relies on two custom code generation modules, the Toolkit and the Kernel Generator, which follow a Model-View-Controller architectural pattern on top of the Jinja2 template engine library. Using Jinja2's templates to abstract the critical components of the engine and generated glue code, we isolate the application development from the engine. The template language also allows us to define and use custom template macros that isolate low-level optimizations from the numerical scheme described in the templates. We present three use cases, each focusing on one of our user roles, showcasing how the design of the code generation modules allows to easily expand the solver schemes to support novel demands from applications, to add optimized algorithmic schemes (with reduced memory footprint, e.g.), or provide improved low-level SIMD vectorization support.

cs.MS↗
arXiv2019

Semi-Automatic Task Graph Construction for $\mathcal{H}$-Matrix Arithmetic

A new method to construct task graphs for \mcH-matrix arithmetic is introduced, which uses the information associated with all tasks of the standard recursive \mcH-matrix algorithms, e.g., the block index set of the matrix blocks involved in the computation. Task refinement, i.e., the replacement of tasks by sub-computations, is then used to proceed in the \mcH-matrix hierarchy until the matrix blocks containing the actual matrix data are reached. This process is a natural extension of the classical, recursive way in which \mcH-matrix arithmetic is defined and thereby simplifies the efficient usage of many-core systems. Examples for standard and accumulator based \mcH-arithmetic are shown for model problems with different block structures.

cs.MS↗
arXiv2019

Abstract Compilation for Verification of Numerical Accuracy Properties

Verification of numerical accuracy properties in modern software remains an important and challenging task. This paper describes an original framework combining different solutions for numerical accuracy. First, we extend an existing runtime verification tool called E-ACSL with rational numbers to monitor accuracy properties at runtime. Second, we present an abstract compiler, FLDCompiler, that performs a source-to-source transformation such that the execution of the resulting program, called an abstract execution, is an abstract interpretation of the initial program. Third, we propose an instrumentation library FLDLib that formally propagates accuracy properties along an abstract execution. While each of these solutions has its own interest, we emphasize the benefits of their combination for an industrial setting. Initial experiments show that the proposed technique can efficiently and soundly analyze the accuracy of industrial programs by restricting the analysis on thin numerical scenarios.

cs.SE↗
arXiv2020

Using performance analysis tools for parallel-in-time integrators -- Does my time-parallel code do what I think it does?

While many ideas and proofs of concept for parallel-in-time integration methods exists, the number of large-scale, accessible time-parallel codes is rather small. This is often due to the apparent or subtle complexity of the algorithms and the many pitfalls awaiting developers of parallel numerical software. One example of such a time-parallel code is pySDC, which implements, among others, the parallel full approximation scheme in space and time (PFASST). Inspired by nonlinear multigrid ideas, PFASST allows to integrate multiple time-steps simultaneously using a space-time hierarchy of spectral deferred corrections. In this paper we demonstrate the application of performance analysis tools to the PFASST implementation pySDC. Tracing the path we took for this work, we highlight the obstacles encountered, describe remedies and explain the sometimes surprising findings made possible by the tools. Although focusing only on a single implementation of a particular parallel-in-time integrator, we hope that our results and in particular the way we obtained them are a blueprint for other time-parallel codes.

cs.PF↗
arXiv2019

PFASST-ER: Combining the Parallel Full Approximation Scheme in Space and Time with parallelization across the method

To extend prevailing scaling limits when solving time-dependent partial differential equations, the parallel full approximation scheme in space and time (PFASST) has been shown to be a promising parallel-in-time integrator. Similar to a space-time multigrid, PFASST is able to compute multiple time-steps simultaneously and is therefore in particular suitable for large-scale applications on high performance computing systems. In this work we couple PFASST with a parallel spectral deferred correction (SDC) method, forming an unprecedented doubly time-parallel integrator. While PFASST provides global, large-scale "parallelization across the step", the inner parallel SDC method allows to integrate each individual time-step "parallel across the method" using a diagonalized local Quasi-Newton solver. This new method, which we call "PFASST with Enhanced concuRrency" (PFASST-ER), therefore exposes even more temporal parallelism. For two challenging nonlinear reaction-diffusion problems, we show that PFASST-ER works more efficiently than the classical variants of PFASST and can be used to run parallel-in-time beyond the number of time-steps.

cs.MS↗
arXiv2019

CheasePy

CheasePy is code written in Python to run the CHEASE (Cubic Hermite Element Axisymmetric Static Equilibrium) code, which solves the Grad-Shafranov equation for toroidal MHD equilibria using pressure and current profiles and fixed plasma boundaries that is defined by a set of experimental data points (R,Z). The CheasePy code allows an iterative running of the CHEASE code either to check the preservation of MHD equilibria or converging to an experimentally defined total toroidal plasma current by modifying any input quantity.

cs.MS↗
arXiv2019

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several common benchmarks.

cs.LG↗
arXiv2019

differint: A Python Package for Numerical Fractional Calculus

Fractional calculus has become widely studied and applied to physical problems in recent years. As a result, many methods for the numerical computation of fractional derivatives and integrals have been defined. However, these algorithms are often programmed in an ad hoc manner, requiring researchers to implement and debug their own code. This work introduces the \textit{differint} software package, which offers a single repository for multiple numerical algorithms for the computation of fractional derivatives and integrals. This package is coded in the open-source Python programming language. The Grünwald-Letnikov, improved Grünwald-Letnikov, and Riemann-Liouville algorithms from the fractional calculus are included in this package. The algorithms presented are computed from their descriptions found in [2]. This work concludes with suggestions for the application of the \textit{differint} software package.

cs.MS↗
arXiv2019

High Accuracy Low Precision QR Factorization and Least Square Solver on GPU with TensorCore

Driven by the insatiable needs to process ever larger amount of data with more complex models, modern computer processors and accelerators are beginning to offer half precision floating point arithmetic support, and extremely optimized special units such as NVIDIA TensorCore on GPU and Google Tensor Processing Unit (TPU) that does half precision matrix-matrix multiplication exceptionally efficiently. In this paper we present a large scale mixed precision linear least square solver that achieves high accuracy using the low precision TensorCore GPU. The mixed precision system consists of both innovative algorithms and implementations, and is shown to be up to 14x faster than single precision cuSOLVER at QR matrix factorization at large scale with slightly lower accuracy, and up to 10x faster than double precision direct QR least square solver with comparable accuracy.

cs.MS↗
arXiv2021

PETSc TSAdjoint: a discrete adjoint ODE solver for first-order and second-order sensitivity analysis

We present a new software system PETSc TSAdjoint for first-order and second-order adjoint sensitivity analysis of time-dependent nonlinear differential equations. The derivative calculation in PETSc TSAdjoint is essentially a high-level algorithmic differentiation process. The adjoint models are derived by differentiating the timestepping algorithms and implemented based on the parallel infrastructure in PETSc. Full differentiation of the library code including MPI routines thus is avoided, and users do not need to derive their own adjoint models for their specific applications. PETSc TSAdjoint can compute the first-order derivative, that is, the gradient of a scalar functional, and the Hessian-vector product that carries second-order derivative information, while requiring minimal input (a few callbacks) from the users. Optimal checkpointing schemes are employed by the adjoint model in a manner that is transparent to users. Usability, efficiency, and scalability are demonstrated through examples from a variety of applications.

cs.MS↗
arXiv2019

Assembly of multiscale linear PDE operators

In numerous applications the mathematical model consists of different processes coupled across a lower dimensional manifold. Due to the multiscale coupling, finite element discretization of such models presents a challenge. Assuming that only singlescale finite element forms can be assembled we present here a simple algorithm for representing multiscale models as linear operators suitable for Krylov methods. Flexibility of the approach is demonstrated by numerical examples with coupling across dimensionality gap 1 and 2. Preconditioners for several of the problems are discussed.

math.NA↗
arXiv2019

AVaN Pack: An Analytical/Numerical Solution for Variance-Based Sensitivity Analysis

Sensitivity analysis is an important concept to analyze the influences of parameters in a system, an equation or a collection of data. The methods used for sensitivity analysis are divided into deterministic and statistical techniques. Generally, deterministic techniques analyze fixed points of a model whilst stochastic techniques analyze a range of values. Deterministic methods fail in analyze the entire range of input values and stochastic methods generate outcomes with random errors. In this manuscript, we are interested in stochastic methods, mainly in variance-based techniques such as Variance and Sobol indices, since this class of techniques is largely used on literature. The objective of this manuscript is to present an analytical solution for variance based sensitive analysis. As a result of this research, two small programs were developed in Javascript named as AVaN Pack (Analysis of Variance through Numerical solution). These programs allow users to find the contribution of each individual parameter in any function by means of a mathematical solution, instead of sampling-based ones.

stat.OT↗
arXiv2019

Strategies for the vectorized Block Conjugate Gradients method

Block Krylov methods have recently gained a lot of attraction. Due to their increased arithmetic intensity they offer a promising way to improve performance on modern hardware. Recently Frommer et al. presented a block Krylov framework that combines the advantages of block Krylov methods and data parallel methods. We review this framework and apply it on the Block Conjugate Gradients method,to solve linear systems with multiple right hand sides. In this course we consider challenges that occur on modern hardware, like a limited memory bandwidth, the use of SIMD instructions and the communication overhead. We present a performance model to predict the efficiency of different Block CG variants and compare these with experimental numerical results.

math.NA↗
arXiv2019

Linnea: Automatic Generation of Efficient Linear Algebra Programs

The translation of linear algebra computations into efficient sequences of library calls is a non-trivial task that requires expertise in both linear algebra and high-performance computing. Almost all high-level languages and libraries for matrix computations (e.g., Matlab, Eigen) internally use optimized kernels such as those provided by BLAS and LAPACK; however, their translation algorithms are often too simplistic and thus lead to a suboptimal use of said kernels, resulting in significant performance losses. In order to combine the productivity offered by high-level languages, and the performance of low-level kernels, we are developing Linnea, a code generator for linear algebra problems. As input, Linnea takes a high-level description of a linear algebra problem; as output, it returns an efficient sequence of calls to high-performance kernels. Linnea uses a custom best-first search algorithm to find a first solution in less than a second, and increasingly better solutions when given more time. In 125 test problems, the code generated by Linnea almost always outperforms Matlab, Julia, Eigen and Armadillo, with speedups up to and exceeding 10x.

cs.MS↗
arXiv2019

A Unified Iteration Space Transformation Framework for Sparse and Dense Tensor Algebra

We address the problem of optimizing mixed sparse and dense tensor algebra in a compiler. We show that standard loop transformations, such as strip-mining, tiling, collapsing, parallelization and vectorization, can be applied to irregular loops over sparse iteration spaces. We also show how these transformations can be applied to the contiguous value arrays of sparse tensor data structures, which we call their position space, to unlock load-balanced tiling and parallelism. We have prototyped these concepts in the open-source TACO system, where they are exposed as a scheduling API similar to the Halide domain-specific language for dense computations. Using this scheduling API, we show how to optimize mixed sparse/dense tensor algebra expressions, how to generate load-balanced code by scheduling sparse tensor algebra in position space, and how to generate sparse tensor algebra GPU code. Our evaluation shows that our transformations let us generate good code that is competitive with many hand-optimized implementations from the literature.

cs.MS↗
arXiv2020

Comparing Python, Go, and C++ on the N-Queens Problem

Python currently is the dominant language in the field of Machine Learning but is often criticized for being slow to perform certain tasks. In this report, we use the well-known $N$-queens puzzle as a benchmark to show that once compiled using the Numba compiler it becomes competitive with C++ and Go in terms of execution speed while still allowing for very fast prototyping. This is true of both sequential and parallel programs. In most cases that arise in an academic environment, it therefore makes sense to develop in ordinary Python, identify computational bottlenecks, and use Numba to remove them.

cs.MS↗
arXiv2020

MonteCarloMeasurements.jl: Nonlinear Propagation of Arbitrary Multivariate Distributions by means of Method Overloading

This manuscript outlines a software package that facilitates working with probability distributions by means of Monte-Carlo methods, in a way that allows for propagation of multivariate probability distributions through arbitrary functions. We provide a \emph{type} that represents probability distributions by an internal vector of unweighted samples, \texttt{Particles}, which is a subtype of a \texttt{Real} number and behaves just like a regular real number in calculations by means of method overloading. This makes the software easy to work with and presents minimal friction for the user. We highlight how this design facilitates optimal usage of SIMD instructions and showcase the package for uncertainty propagation through an off-the-shelf ODE solver, as well as for robust probabilistic optimization with automatic differentiation.

cs.MS↗