SEARCH · arXiv Science

Results for “cs.MS”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30Linked to original sources

arXiv2016

jInv -- a flexible Julia package for PDE parameter estimation

Estimating parameters of Partial Differential Equations (PDEs) from noisy and indirect measurements often requires solving ill-posed inverse problems. These so called parameter estimation or inverse medium problems arise in a variety of applications such as geophysical, medical imaging, and nondestructive testing. Their solution is computationally intense since the underlying PDEs need to be solved numerous times until the reconstruction of the parameters is sufficiently accurate. Typically, the computational demand grows significantly when more measurements are available, which poses severe challenges to inversion algorithms as measurement devices become more powerful. In this paper we present jInv, a flexible framework and open source software that provides parallel algorithms for solving parameter estimation problems with many measurements. Being written in the expressive programming language Julia, jInv is portable, easy to understand and extend, cross-platform tested, and well-documented. It provides novel parallelization schemes that exploit the inherent structure of many parameter estimation problems and can be used to solve multiphysics inversion problems as is demonstrated using numerical experiments motivated by geophysical imaging.

cs.MS↗

arXiv2017

Design of a high-performance GEMM-like Tensor-Tensor Multiplication

We present "GEMM-like Tensor-Tensor multiplication" (GETT), a novel approach to tensor contractions that mirrors the design of a high-performance general matrix-matrix multiplication (GEMM). The critical insight behind GETT is the identification of three index sets, involved in the tensor contraction, which enable us to systematically reduce an arbitrary tensor contraction to loops around a highly tuned "macro-kernel". This macro-kernel operates on suitably prepared ("packed") sub-tensors that reside in a specified level of the cache hierarchy. In contrast to previous approaches to tensor contractions, GETT exhibits desirable features such as unit-stride memory accesses, cache-awareness, as well as full vectorization, without requiring auxiliary memory. To compare our technique with other modern tensor contractions, we integrate GETT alongside the so called Transpose-Transpose-GEMM-Transpose and Loops-over-GEMM approaches into an open source "Tensor Contraction Code Generator" (TCCG). The performance results for a wide range of tensor contractions suggest that GETT has the potential of becoming the method of choice: While GETT exhibits excellent performance across the board, its effectiveness for bandwidth-bound tensor contractions is especially impressive, outperforming existing approaches by up to $12.4\times$. More precisely, GETT achieves speedups of up to $1.41\times$ over an equivalent-sized GEMM for bandwidth-bound tensor contractions while attaining up to $91.3\%$ of peak floating-point performance for compute-bound tensor contractions.

cs.MS↗

arXiv2016

Using the pyMIC Offload Module in PyFR

PyFR is an open-source high-order accurate computational fluid dynamics solver for unstructured grids. It is designed to efficiently solve the compressible Navier-Stokes equations on a range of hardware platforms, including GPUs and CPUs. In this paper we will describe how the Python Offload Infrastructure for the Intel Many Integrated Core Architecture (pyMIC) was used to enable PyFR to run with near native performance on the Intel Xeon Phi coprocessor. We will introduce the architecture of both pyMIC and PyFR and present a variety of examples showcasing the capabilities of pyMIC. Further, we will also compare the contrast pyMIC to other approaches including native execution and OpenCL. The process of adding support for pyMIC into PyFR will be described in detail. Benchmark results show that for a standard cylinder flow problem PyFR with pyMIC is able achieve 240 GFLOP/s of sustained double precision floating point performance; for a 1.85 times improvement over PyFR with C/OpenMP on a 12 core Intel Xeon E5-2697 v2 CPU.

cs.MS↗

arXiv2016

Massively parallel implementation in Python of a pseudo-spectral DNS code for turbulent flows

Direct Numerical Simulations (DNS) of the Navier Stokes equations is a valuable research tool in fluid dynamics, but there are very few publicly available codes and, due to heavy number crunching, codes are usually written in low-level languages. In this work a \textasciitilde{}100 line standard scientific Python DNS code is described that nearly matches the performance of pure C for thousands of processors and billions of unknowns. With optimization of a few routines in Cython, it is found to match the performance of a more or less identical solver implemented from scratch in C++. Keys to the efficiency of the solver are the mesh decomposition and three dimensional FFT routines, implemented directly in Python using MPI, wrapped through MPI for Python, and a serial FFT module (both numpy.fft or pyFFTW may be used). Two popular decomposition strategies, slab and pencil, have been implemented and tested.

cs.MS↗

arXiv2017

PRIMME_SVDS: A High-Performance Preconditioned SVD Solver for Accurate Large-Scale Computations

The increasing number of applications requiring the solution of large scale singular value problems have rekindled interest in iterative methods for the SVD. Some promising recent ad- vances in large scale iterative methods are still plagued by slow convergence and accuracy limitations for computing smallest singular triplets. Furthermore, their current implementations in MATLAB cannot address the required large problems. Recently, we presented a preconditioned, two-stage method to effectively and accurately compute a small number of extreme singular triplets. In this research, we present a high-performance software, PRIMME SVDS, that implements our hybrid method based on the state-of-the-art eigensolver package PRIMME for both largest and smallest singular values. PRIMME SVDS fills a gap in production level software for computing the partial SVD, especially with preconditioning. The numerical experiments demonstrate its superior performance compared to other state-of-the-art software and its good parallel performance under strong and weak scaling.

cs.MS↗

arXiv2016

Accelerating eigenvector and pseudospectra computation using blocked multi-shift triangular solves

Multi-shift triangular solves are basic linear algebra calculations with applications in eigenvector and pseudospectra computation. We propose blocked algorithms that efficiently exploit Level 3 BLAS to perform multi-shift triangular solves and safe multi-shift triangular solves. Numerical experiments indicate that computing triangular eigenvectors with a safe multi-shift triangular solve achieves speedups by a factor of 60 relative to LAPACK. This algorithm accelerates the calculation of general eigenvectors threefold. When using multi-shift triangular solves to compute pseudospectra, we report ninefold speedups relative to EigTool.

cs.MS↗

arXiv2016

Form Follows Function -- Do algorithms and applications challenge or drag behind the hardware evolution?

We summarise some of the key statements made at the workshop Form Follows Function at ISC High Performance 2016. The summary highlights what type of co-design the presented projects experience; often in the absence of an explicit co-design agenda. Their software development picks up hardware trends but it also influences the hardware development. Observations illustrate that this cycle not always is optimal for both sides as it is not proactively steered. Key statements characterise ideas how it might be possible to integrate both hardware and software creation closer to the best of both worlds---again even without classic co-design in mind where new pieces of hardware are created. The workshop finally identified three development idioms that might help to improve software and system design with respect to emerging hardware.

cs.MS↗

arXiv2016

The Vectorization of the Tersoff Multi-Body Potential: An Exercise in Performance Portability

Molecular dynamics simulations, an indispensable research tool in computational chemistry and materials science, consume a significant portion of the supercomputing cycles around the world. We focus on multi-body potentials and aim at achieving performance portability. Compared with well-studied pair potentials, multibody potentials deliver increased simulation accuracy but are too complex for effective compiler optimization. Because of this, achieving cross-platform performance remains an open question. By abstracting from target architecture and computing precision, we develop a vectorization scheme applicable to both CPUs and accelerators. We present results for the Tersoff potential within the molecular dynamics code LAMMPS on several architectures, demonstrating efficiency gains not only for computational kernels, but also for large-scale simulations. On a cluster of Intel Xeon Phi's, our optimized solver is between 3 and 5 times faster than the pure MPI reference.

cs.CE↗

arXiv2016

Scheduling massively parallel multigrid for multilevel Monte Carlo methods

The computational complexity of naive, sampling-based uncertainty quantification for 3D partial differential equations is extremely high. Multilevel approaches, such as multilevel Monte Carlo (MLMC), can reduce the complexity significantly, but to exploit them fully in a parallel environment, sophisticated scheduling strategies are needed. Often fast algorithms that are executed in parallel are essential to compute fine level samples in 3D, whereas to compute individual coarse level samples only moderate numbers of processors can be employed efficiently. We make use of multiple instances of a parallel multigrid solver combined with advanced load balancing techniques. In particular, we optimize the concurrent execution across the three layers of the MLMC method: parallelization across levels, across samples, and across the spatial grid. The overall efficiency and performance of these methods will be analyzed. Here the scalability window of the multigrid solver is revealed as being essential, i.e., the property that the solution can be computed with a range of process numbers while maintaining good parallel efficiency. We evaluate the new scheduling strategies in a series of numerical tests, and conclude the paper demonstrating large 3D scaling experiments.

cs.CE↗

arXiv2016

Generalized Sampling in Julia

Generalized sampling is a numerically stable framework for obtaining reconstructions of signals in different bases and frames from their samples. In this paper, we will introduce a carefully documented toolbox for performing generalized sampling in Julia. Julia is a new language for technical computing with focus on performance, which is ideally suited to handle the large size problems often encountered in generalized sampling. The toolbox provides specialized solutions for the setup of Fourier bases and wavelets. The performance of the toolbox is compared to existing implementations of generalized sampling in MATLAB.

cs.MS↗

arXiv2016

Finite Element Integration with Quadrature on the GPU

We present a novel, quadrature-based finite element integration method for low-order elements on GPUs, using a pattern we call \textit{thread transposition} to avoid reductions while vectorizing aggressively. On the NVIDIA GTX580, which has a nominal single precision peak flop rate of 1.5 TF/s and a memory bandwidth of 192 GB/s, we achieve close to 300 GF/s for element integration on first-order discretization of the Laplacian operator with variable coefficients in two dimensions, and over 400 GF/s in three dimensions. From our performance model we find that this corresponds to 90\% of our measured achievable bandwidth peak of 310 GF/s. Further experimental results also match the predicted performance when used with double precision (120 GF/s in two dimensions, 150 GF/s in three dimensions). Results obtained for the linear elasticity equations (220 GF/s and 70 GF/s in two dimensions, 180 GF/s and 60 GF/s in three dimensions) also demonstrate the applicability of our method to vector-valued partial differential equations.

cs.MS↗

arXiv2016

Optimized Automatic Code Generation for Geometric Algebra Based Algorithms with Ray Tracing Application

Automatic code generation for low-dimensional geometric algorithms is capable of producing efficient low-level software code through a high-level geometric domain specific language. Geometric Algebra (GA) is one of the most suitable algebraic systems for being the base for such code generator. This work presents an attempt at realizing such idea in practice. A novel GA-based geometric code generator, called GMac, is proposed. Comparisons to similar GA-based code generators are provided. The possibility of fully benefiting from the symbolic power of GA while obtaining good performance and maintainability of software implementations is illustrated through a ray tracing application.

cs.MS↗

arXiv2016

Forward-Mode Automatic Differentiation in Julia

We present ForwardDiff, a Julia package for forward-mode automatic differentiation (AD) featuring performance competitive with low-level languages like C++. Unlike recently developed AD tools in other popular high-level languages such as Python and MATLAB, ForwardDiff takes advantage of just-in-time (JIT) compilation to transparently recompile AD-unaware user code, enabling efficient support for higher-order differentiation and differentiation using custom number types (including complex numbers). For gradient and Jacobian calculations, ForwardDiff provides a variant of vector-forward mode that avoids expensive heap allocation and makes better use of memory bandwidth than traditional vector mode. In our numerical experiments, we demonstrate that for nontrivially large dimensions, ForwardDiff's gradient computations can be faster than a reverse-mode implementation from the Python-based autograd package. We also illustrate how ForwardDiff is used effectively within JuMP, a modeling language for optimization. According to our usage statistics, 41 unique repositories on GitHub depend on ForwardDiff, with users from diverse fields such as astronomy, optimization, finite element analysis, and statistics. This document is an extended abstract that has been accepted for presentation at the AD2016 7th International Conference on Algorithmic Differentiation.

cs.MS↗

arXiv2016

An Asynchronous Task-based Fan-Both Sparse Cholesky Solver

Systems of linear equations arise at the heart of many scientific and engineering applications. Many of these linear systems are sparse; i.e., most of the elements in the coefficient matrix are zero. Direct methods based on matrix factorizations are sometimes needed to ensure accurate solutions. For example, accurate solution of sparse linear systems is needed in shift-invert Lanczos to compute interior eigenvalues. The performance and resource usage of sparse matrix factorizations are critical to time-to-solution and maximum problem size solvable on a given platform. In many applications, the coefficient matrices are symmetric, and exploiting symmetry will reduce both the amount of work and storage cost required for factorization. When the factorization is performed on large-scale distributed memory platforms, communication cost is critical to the performance of the algorithm. At the same time, network topologies have become increasingly complex, so that modern platforms exhibit a high level of performance variability. This makes scheduling of computations an intricate and performance-critical task. In this paper, we investigate the use of an asynchronous task paradigm, one-sided communication and dynamic scheduling in implementing sparse Cholesky factorization (symPACK) on large-scale distributed memory platforms. Our solver symPACK relies on efficient and flexible communication primitives provided by the UPC++ library. Performance evaluation shows good scalability and that symPACK outperforms state-of-the-art parallel distributed memory factorization packages, validating our approach on practical cases.

cs.MS↗

arXiv2016

An exact, cache-localized algorithm for the sub-quadratic convolution of hypercubes

Fast multidimensional convolution can be performed naively in quadratic time and can often be performed more efficiently via the Fourier transform; however, when the dimensionality is large, these algorithms become more challenging. A method is proposed for performing exact hypercube convolution in sub-quadratic time. The method outperforms FFTPACK, called via numpy, and FFTW, called via pyfftw) for hypercube convolution. Embeddings in hypercubes can be paired with sub-quadratic hypercube convolution method to construct sub-quadratic algorithms for variants of vector convolution.

cs.CG↗

arXiv2016

Computation of the incomplete gamma function for negative values of the argument

An algorithm for computing the incomplete gamma function $γ^*(a,z)$ for real values of the parameter $a$ and negative real values of the argument $z$ is presented. The algorithm combines the use of series expansions, Poincaré-type expansions, uniform asymptotic expansions and recurrence relations, depending on the parameter region. A relative accuracy $\sim 10^{-13}$ in the parameter region $(a,z) \in [-500,\,500] \times [-500,\,0)$ can be obtained when computing the function $γ^*(a,z)$ with the Fortran 90 module IncgamNEG implementing the algorithm.

cs.MS↗

arXiv2016

A Functional Package for Automatic Solution of Ordinary Differential Equations with Spectral Methods

We present a Python module named PyCheb, to solve the ordinary differential equations by using spectral collocation method. PyCheb incorporates discretization using Chebyshev points, barycentric interpolation and iterate methods. With this Python module, users can initialize the ODEsolver class by passing attributes, including the both sides of a given differential equation, boundary conditions, and the number of Chebyshev points, which can also be generated automatically by the ideal precision, to the constructor of ODEsolver class. Then, the instance of the ODEsolver class can be used to automatically determine the resolution of the differential equation as well as generate the graph of the high-precision approximate solution. (If you have any questions, please send me an email and I will reply ASAP. e-mail:shaohui_liu@qq.com/2013141482143@stu.scu.edu.cn)

cs.MS↗

arXiv2016

Devito: automated fast finite difference computation

Domain specific languages have successfully been used in a variety of fields to cleanly express scientific problems as well as to simplify implementation and performance opti- mization on different computer architectures. Although a large number of stencil languages are available, finite differ- ence domain specific languages have proved challenging to design because most practical use cases require additional features that fall outside the finite difference abstraction. Inspired by the complexity of real-world seismic imaging problems, we introduce Devito, a domain specific language in which high level equations are expressed using symbolic expressions from the SymPy package. Complex equations are automatically manipulated, optimized, and translated into highly optimized C code that aims to perform compa- rably or better than hand-tuned code. All this is transpar- ent to users, who only see concise symbolic mathematical expressions.

cs.MS↗