SEARCH · arXiv Science

Results for “cs.MS”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39Linked to original sources

arXiv2018

Certified Roundoff Error Bounds using Bernstein Expansions and Sparse Krivine-Stengle Representations

Floating point error is a drawback of embedded systems implementation that is difficult to avoid. Computing rigorous upper bounds of roundoff errors is absolutely necessary for the validation of critical software. This problem of computing rigorous upper bounds is even more challenging when addressing non-linear programs. In this paper, we propose and compare two new algorithms based on Bernstein expansions and sparse Krivine-Stengle representations, adapted from the field of the global optimization, to compute upper bounds of roundoff errors for programs implementing polynomial and rational functions. We also provide the convergence rate of these two algorithms. We release two related software package FPBern and FPKriSten, and compare them with the state-of-the-art tools. We show that these two methods achieve competitive performance, while providing accurate upper bounds by comparison with the other tools.

math.NA↗

arXiv2018

Compiling Diderot: From Tensor Calculus to C

Diderot is a parallel domain-specific language for analysis and visualization of multidimensional scientific images, such as those produced by CT and MRI scanners. In particular, it supports algorithms where tensor fields (i.e., functions from 3D points to tensor values) are used to represent the underlying physical objects that were scanned by the imaging device. Diderot supports higher-order programming where tensor fields are first-class values and where differential operators and lifted linear-algebra operators can be used to express mathematical reasoning directly in the language. While such lifted field operations are central to the definition and computation of many scientific visualization algorithms, to date they have required extensive manual derivations and laborious implementation. The challenge for the Diderot compiler is to effectively translate the high-level mathematical concepts that are expressible in the surface language to a low-level and efficient implementation in C. This paper describes our approach to this challenge, which is based around the careful design of an intermediate representation (IR), called EIN, and a number of compiler transformations that lower the program from tensor calculus to C while avoiding combinatorial explosion in the size of the IR. We describe the challenges in compiling a language like Diderot, the design of EIN, and the transformation used by the compiler. We also present an evaluation of EIN with respect to both compiler efficiency and quality of generated code.

cs.PL↗

arXiv2018

Comparative study of finite element methods using the Time-Accuracy-Size (TAS) spectrum analysis

We present a performance analysis appropriate for comparing algorithms using different numerical discretizations. By taking into account the total time-to-solution, numerical accuracy with respect to an error norm, and the computation rate, a cost-benefit analysis can be performed to determine which algorithm and discretization are particularly suited for an application. This work extends the performance spectrum model in Chang et. al. 2017 for interpretation of hardware and algorithmic tradeoffs in numerical PDE simulation. As a proof-of-concept, popular finite element software packages are used to illustrate this analysis for Poisson's equation.

cs.MS↗

arXiv2018

Numerical integration in arbitrary-precision ball arithmetic

We present an implementation of arbitrary-precision numerical integration with rigorous error bounds in the Arb library. Rapid convergence is ensured for piecewise complex analytic integrals by use of the Petras algorithm, which combines adaptive bisection with adaptive Gaussian quadrature where error bounds are determined via complex magnitudes without evaluating derivatives. The code is general, easy to use, and efficient, often outperforming existing non-rigorous software.

cs.MS↗

arXiv2018

The iisignature library: efficient calculation of iterated-integral signatures and log signatures

Iterated-integral signatures and log signatures are vectors calculated from a path that characterise its shape. They come from the theory of differential equations driven by rough paths, and also have applications in statistics and machine learning. We present algorithms for efficiently calculating these signatures, and benchmark their performance. We release the methods as a Python package.

cs.DS↗

arXiv2018

Moore: Interval Arithmetic in C++20

This article presents the Moore library for interval arithmetic in C++20. It gives examples of how the library can be used, and explains the basic principles underlying its design.

cs.MS↗

arXiv2018

Sparse Tensor Algebra Optimizations with Workspaces

This paper shows how to optimize sparse tensor algebraic expressions by introducing temporary tensors, called workspaces, into the resulting loop nests. We develop a new intermediate language for tensor operations called concrete index notation that extends tensor index notation. Concrete index notation expresses when and where sub-computations occur and what tensor they are stored into. We then describe the workspace optimization in this language, and how to compile it to sparse code by building on prior work in the literature. We demonstrate the importance of the optimization on several important sparse tensor kernels, including sparse matrix-matrix multiplication (SpMM), sparse tensor addition (SpAdd), and the matricized tensor times Khatri-Rao product (MTTKRP) used to factorize tensors. Our results show improvements over prior work on tensor algebra compilation and brings the performance of these kernels on par with state-of-the-art hand-optimized implementations. For example, SpMM was not supported by prior tensor algebra compilers, the performance of MTTKRP on the nell-2 data set improves by 35%, and MTTKRP can for the first time have sparse results.

cs.MS↗

arXiv2018

OpenMath and SMT-LIB

OpenMath and SMT-LIB are languages with very different origins, but both "represent mathematics". We describe SMT-LIB for the OpenMath community and consider adaptations for both languages to support the growing SC-Square initiative.

cs.SC↗

arXiv2018

Scaling Structured Multigrid to 500K+ Cores through Coarse-Grid Redistribution

The efficient solution of sparse, linear systems resulting from the discretization of partial differential equations is crucial to the performance of many physics-based simulations. The algorithmic optimality of multilevel approaches for common discretizations makes them a good candidate for an efficient parallel solver. Yet, modern architectures for high-performance computing systems continue to challenge the parallel scalability of multilevel solvers. While algebraic multigrid methods are robust for solving a variety of problems, the increasing importance of data locality and cost of data movement in modern architectures motivates the need to carefully exploit structure in the problem. Robust logically structured variational multigrid methods, such as Black Box Multigrid (BoxMG), maintain structure throughout the multigrid hierarchy. This avoids indirection and increased coarse-grid communication costs typical in parallel algebraic multigrid. Nevertheless, the parallel scalability of structured multigrid is challenged by coarse-grid problems where the overhead in communication dominates computation. In this paper, an algorithm is introduced for redistributing coarse-grid problems through incremental agglomeration. Guided by a predictive performance model, this algorithm provides robust redistribution decisions for structured multilevel solvers. A two-dimensional diffusion problem is used to demonstrate the significant gain in performance of this algorithm over the previous approach that used agglomeration to one processor. In addition, the parallel scalability of this approach is demonstrated on two large-scale computing systems, with solves on up to 500K+ cores.

cs.MS↗

arXiv2018

Algorithmic Differentiation for Domain Specific Languages

Algorithmic Differentiation (AD) can be used to automate the generation of derivatives in arbitrary software projects. This will generate maintainable derivatives, that are always consistent with the computation of the software. If a domain specific language (DSL) is used in a software the state of the art approach is to differentiate the DSL library with the same AD tool. The drawback of this solution is the reduced performance since the compiler is no longer able to optimize the e.g. SIMD operations. The new approach in this paper integrates the types and operations of the DSL into the AD tool. It will be an operator overloading tool that is generated from an abstract definition of a DSL. This approach enables the compiler to optimize again e.g. for SIMD operation since all calculations are still performed with the original data types. This will also reduce the required memory for AD since the statements inside the DLS implementation are no longer seen by the AD tool. The implementation is presented in the paper and first results for the performance of the solution are presented.

cs.MS↗

arXiv2018

Efficient Realization of Givens Rotation through Algorithm-Architecture Co-design for Acceleration of QR Factorization

We present efficient realization of Generalized Givens Rotation (GGR) based QR factorization that achieves 3-100x better performance in terms of Gflops/watt over state-of-the-art realizations on multicore, and General Purpose Graphics Processing Units (GPGPUs). GGR is an improvement over classical Givens Rotation (GR) operation that can annihilate multiple elements of rows and columns of an input matrix simultaneously. GGR takes 33% lesser multiplications compared to GR. For custom implementation of GGR, we identify macro operations in GGR and realize them on a Reconfigurable Data-path (RDP) tightly coupled to pipeline of a Processing Element (PE). In PE, GGR attains speed-up of 1.1x over Modified Householder Transform (MHT) presented in the literature. For parallel realization of GGR, we use REDEFINE, a scalable massively parallel Coarse-grained Reconfigurable Architecture, and show that the speed-up attained is commensurate with the hardware resources in REDEFINE. GGR also outperforms General Matrix Multiplication (gemm) by 10% in-terms of Gflops/watt which is counter-intuitive.

cs.DC↗

arXiv2018

Glyph: Symbolic Regression Tools

We present Glyph - a Python package for genetic programming based symbolic regression. Glyph is designed for usage let by numerical simulations let by real world experiments. For experimentalists, glyph-remote provides a separation of tasks: a ZeroMQ interface splits the genetic programming optimization task from the evaluation of an experimental (or numerical) run. Glyph can be accessed at http://github.com/ambrosys/glyph . Domain experts are be able to employ symbolic regression in their experiments with ease, even if they are not expert programmers. The reuse potential is kept high by a generic interface design. Glyph is available on PyPI and Github.

cs.MS↗

arXiv2018

PyGOM - A Python Package for Simplifying Modelling with Systems of Ordinary Differential Equations

Ordinary Differential Equations (ODE) are used throughout science where the capture of rates of change in states is sought. While both pieces of commercial and open software exist to study such systems, their efficient and accurate usage frequently requires deep understanding of mathematics and programming. The package we present here, PyGOM, seeks to remove these obstacles for models based on ODE systems. We provide a simple interface for the construction of such systems backed by a comprehensive and easy to use tool--box. This tool--box implements functions to easily perform common operations for ODE systems such as solving, parameter estimation, and stochastic simulation. The package source is freely available and organized in a way that permits easy extension. With both the algebraic and numeric calculations performed automatically (but still accessible), the end user is freed to focus on model development.

cs.MS↗

arXiv2018

Automatic symbolic computation for discontinuous Galerkin finite element methods

The implementation of discontinuous Galerkin finite element methods (DGFEMs) represents a very challenging computational task, particularly for systems of coupled nonlinear PDEs, including multiphysics problems, whose parameters may consist of power series or functionals of the solution variables. Thereby, the exploitation of symbolic algebra to express a given DGFEM approximation of a PDE problem within a high level language, whose syntax closely resembles the mathematical definition, is an invaluable tool. Indeed, this then facilitates the automatic assembly of the resulting system of (nonlinear) equations, as well as the computation of Fréchet derivative(s) of the DGFEM scheme, needed, for example, within a Newton-type solver. However, even exploiting symbolic algebra, the discretisation of coupled systems of PDEs can still be extremely verbose and hard to debug. Thereby, in this article we develop a further layer of abstraction by designing a class structure for the automatic computation of DGFEM formulations. This work has been implemented within the FEniCS package, based on exploiting the Unified Form Language. Numerical examples are presented which highlight the simplicity of implementation of DGFEMs for the numerical approximation of a range of PDE problems.

math.NA↗

arXiv2018

A Blackbox Polynomial System Solver on Parallel Shared Memory Computers

A numerical irreducible decomposition for a polynomial system provides representations for the irreducible factors of all positive dimensional solution sets of the system, separated from its isolated solutions. Homotopy continuation methods are applied to compute a numerical irreducible decomposition. Load balancing and pipelining are techniques in a parallel implementation on a computer with multicore processors. The application of the parallel algorithms is illustrated on solving the cyclic $n$-roots problems, in particular for $n = 8, 9$, and~12.

cs.MS↗

arXiv2019

A Scalable Shared-Memory Parallel Simplex for Large-Scale Linear Programming

The Simplex tableau has been broadly used and investigated in the industry and academia. With the advent of the big data era, ever larger problems are posed to be solved in ever larger machines whose architecture type did not exist in the conception of this algorithm. In this paper, we present a shared-memory parallel implementation of the Simplex tableau algorithm for dense large-scale Linear Programming (LP) problems for use in modern multi-core architectures. We present the general scheme and explain the strategies taken to parallelize each step of the standard simplex algorithm, emphasizing the solutions found to solve performance bottlenecks. We analyzed the speedup and the parallel efficiency for the proposed implementation relative to the standard Simplex algorithm using a shared-memory system with 64 processing cores. The experiments were performed for several different problems, with up to 8192 variables and constraints, in their primal and dual formulations. The results show that the performance is mostly much better when we use the formulation with more variables than inequality constraints. Also, they show that the parallelization strategies applied to avoid bottlenecks lead the implementation to scale well with the problem size and the core count up to a certain limit of problem size. Further analysis showed that this scaling limit was an effect of resource limitation. Even though, our implementation was able to reach speedups in the order of 19x.

cs.DC↗

arXiv2018

μ-cuDNN: Accelerating Deep Learning Frameworks with Micro-Batching

NVIDIA cuDNN is a low-level library that provides GPU kernels frequently used in deep learning. Specifically, cuDNN implements several equivalent convolution algorithms, whose performance and memory footprint may vary considerably, depending on the layer dimensions. When an algorithm is automatically selected by cuDNN, the decision is performed on a per-layer basis, and thus it often resorts to slower algorithms that fit the workspace size constraints. We present μ-cuDNN, a transparent wrapper library for cuDNN, which divides layers' mini-batch computation into several micro-batches. Based on Dynamic Programming and Integer Linear Programming, μ-cuDNN enables faster algorithms by decreasing the workspace requirements. At the same time, μ-cuDNN keeps the computational semantics unchanged, so that it decouples statistical efficiency from the hardware efficiency safely. We demonstrate the effectiveness of μ-cuDNN over two frameworks, Caffe and TensorFlow, achieving speedups of 1.63x for AlexNet and 1.21x for ResNet-18 on P100-SXM2 GPU. These results indicate that using micro-batches can seamlessly increase the performance of deep learning, while maintaining the same memory footprint.

cs.LG↗

arXiv2018

Adaptive control in rollforward recovery for extreme scale multigrid

With the increasing number of compute components, failures in future exa-scale computer systems are expected to become more frequent. This motivates the study of novel resilience techniques. Here, we extend a recently proposed algorithm-based recovery method for multigrid iterations by introducing an adaptive control. After a fault, the healthy part of the system continues the iterative solution process, while the solution in the faulty domain is re-constructed by an asynchronous on-line recovery. The computations in both the faulty and healthy subdomains must be coordinated in a sensitive way, in particular, both under and over-solving must be avoided. Both of these waste computational resources and will therefore increase the overall time-to-solution. To control the local recovery and guarantee an optimal re-coupling, we introduce a stopping criterion based on a mathematical error estimator. It involves hierarchical weighted sums of residuals within the context of uniformly refined meshes and is well-suited in the context of parallel high-performance computing. The re-coupling process is steered by local contributions of the error estimator. We propose and compare two criteria which differ in their weights. Failure scenarios when solving up to $6.9\cdot10^{11}$ unknowns on more than 245\,766 parallel processes will be reported on a state-of-the-art peta-scale supercomputer demonstrating the robustness of the method.

cs.MS↗