SEARCH · arXiv Science

Results for “cs.MS”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34Linked to original sources

arXiv2017

Two variants of the Froiduire-Pin Algorithm for finite semigroups

In this paper, we present two algorithms based on the Froidure-Pin Algorithm for computing the structure of a finite semigroup from a generating set. As was the case with the original algorithm of Froidure and Pin, the algorithms presented here produce the left and right Cayley graphs, a confluent terminating rewriting system, and a reduced word of the rewriting system for every element of the semigroup. If $U$ is any semigroup, and $A$ is a subset of $U$, then we denote by $\langle A\rangle$ the least subsemigroup of $U$ containing $A$. If $B$ is any other subset of $U$, then, roughly speaking, the first algorithm we present describes how to use any information about $\langle A\rangle$, that has been found using the Froidure-Pin Algorithm, to compute the semigroup $\langle A\cup B\rangle$. More precisely, we describe the data structure for a finite semigroup $S$ given by Froidure and Pin, and how to obtain such a data structure for $\langle A\cup B\rangle$ from that for $\langle A\rangle$. The second algorithm is a lock-free concurrent version of the Froidure-Pin Algorithm.

math.GR↗

arXiv2017

HPTT: A High-Performance Tensor Transposition C++ Library

Recently we presented TTC, a domain-specific compiler for tensor transpositions. Despite the fact that the performance of the generated code is nearly optimal, due to its offline nature, TTC cannot be utilized in all the application codes in which the tensor sizes and the necessary tensor permutations are determined at runtime. To overcome this limitation, we introduce the open-source C++ library High-Performance Tensor Transposition (HPTT). Similar to TTC, HPTT incorporates optimizations such as blocking, multi-threading, and explicit vectorization; furthermore it decomposes any transposition into multiple loops around a so called micro-kernel. This modular design---inspired by BLIS---makes HPTT easy to port to different architectures, by only replacing the hand-vectorized micro-kernel (e.g., a 4x4 transpose). HPTT also offers an optional autotuning framework---guided by a performance model---that explores a vast search space of implementations at runtime (similar to FFTW). Across a wide range of different tensor transpositions and architectures (e.g., Intel Ivy Bridge, Intel Knights Landing, ARMv7, IBM Power7), HPTT attains a bandwidth comparable to that of SAXPY, and yields remarkable speedups over Eigen's tensor transposition implementation. Most importantly, the integration of HPTT into the Cyclops Tensor Framework (CTF) improves the overall performance of tensor contractions by up to 3.1x.

cs.MS↗

arXiv2018

DATeS: A Highly-Extensible Data Assimilation Testing Suite v1.0

A flexible and highly-extensible data assimilation testing suite, named DATeS, is described in this paper. DATeS aims to offer a unified testing environment that allows researchers to compare different data assimilation methodologies and understand their performance in various settings. The core of DATeS is implemented in Python and takes advantage of its object-oriented capabilities. The main components of the package (the numerical models, the data assimilation algorithms, the linear algebra solvers, and the time discretization routines) are independent of each other, which offers great flexibility to configure data assimilation applications. DATeS can interface easily with large third-party numerical models written in Fortran or in C, and with a plethora of external solvers.

cs.MS↗

arXiv2017

Computing Tropical Prevarieties in Parallel

The computation of the tropical prevariety is the first step in the application of polyhedral methods to compute positive dimensional solution sets of polynomial systems. In particular, pretropisms are candidate leading exponents for the power series developments of the solutions. The computation of the power series may start as soon as one pretropism is available, so our parallel computation of the tropical prevariety has an application in a pipelined solver. We present a parallel implementation of dynamic enumeration. Our first distributed memory implementation with forked processes achieved good speedups, but quite often resulted in large variations in the execution times of the processes. The shared memory multithreaded version applies work stealing to reduce the variability of the run time. Our implementation applies the thread safe Parma Polyhedral Library (PPL), in exact arithmetic with the GNU Multiprecision Arithmetic Library (GMP), aided by the fast memory allocations of TCMalloc. Our parallel implementation is capable of computing the tropical prevariety of the cyclic 16-roots problem. We also report on computational experiments on the $n$-body and $n$-vortex problems; our computational results compare favorably with Gfan.

cs.MS↗

arXiv2017

cuTT: A High-Performance Tensor Transpose Library for CUDA Compatible GPUs

We introduce the CUDA Tensor Transpose (cuTT) library that implements high-performance tensor transposes for NVIDIA GPUs with Kepler and above architectures. cuTT achieves high performance by (a) utilizing two GPU-optimized transpose algorithms that both use a shared memory buffer in order to reduce global memory access scatter, and by (b) computing memory positions of tensor elements using a thread-parallel algorithm. We evaluate the performance of cuTT on a variety of benchmarks with tensor ranks ranging from 2 to 12 and show that cuTT performance is independent of the tensor rank and that it performs no worse than an approach based on code generation. We develop a heuristic scheme for choosing the optimal parameters for tensor transpose algorithms by implementing an analytical GPU performance model that can be used at runtime without need for performance measurements or profiling. Finally, by integrating cuTT into the tensor algebra library TAL-SH, we significantly reduce the tensor transpose overhead in tensor contractions, achieving as low as just one percent overhead for arithmetically intensive tensor contractions.

cs.MS↗

arXiv2017

Computing the Lambert W function in arbitrary-precision complex interval arithmetic

We describe an algorithm to evaluate all the complex branches of the Lambert W function with rigorous error bounds in interval arithmetic, which has been implemented in the Arb library. The classic 1996 paper on the Lambert W function by Corless et al. provides a thorough but partly heuristic numerical analysis which needs to be complemented with some explicit inequalities and practical observations about managing precision and branch cuts.

cs.MS↗

arXiv2017

A performance spectrum for parallel computational frameworks that solve PDEs

Important computational physics problems are often large-scale in nature, and it is highly desirable to have robust and high performing computational frameworks that can quickly address these problems. However, it is no trivial task to determine whether a computational framework is performing efficiently or is scalable. The aim of this paper is to present various strategies for better understanding the performance of any parallel computational frameworks for solving PDEs. Important performance issues that negatively impact time-to-solution are discussed, and we propose a performance spectrum analysis that can enhance one's understanding of critical aforementioned performance issues. As proof of concept, we examine commonly used finite element simulation packages and software and apply the performance spectrum to quickly analyze the performance and scalability across various hardware platforms, software implementations, and numerical discretizations. It is shown that the proposed performance spectrum is a versatile performance model that is not only extendable to more complex PDEs such as hydrostatic ice sheet flow equations, but also useful for understanding hardware performance in a massively parallel computing environment. Potential applications and future extensions of this work are also discussed.

cs.MS↗

arXiv2017

Spin Summations: A High-Performance Perspective

Besides tensor contractions, one of the most pronounced computational bottlenecks in the non-orthogonally spin-adapted forms of the quantum chemistry methods CCSDT and CCSDTQ, and their approximate forms---including CCSD(T) and CCSDT(Q)---are spin summations. At a first sight, spin summations are operations similar to tensor transpositions; a closer look instead reveals additional challenges to high-performance calculations, including temporal locality as well as scattered memory accesses. This publication explores a sequence of algorithmic solutions for spin summations, each exploiting individual properties of either the underlying hardware (e.g. caches, vectorization), or the problem itself (e.g. factorizability). The final algorithm combines the advantages of all the solutions, while avoiding their drawbacks; this algorithm, achieves high-performance through parallelization, vectorization, and by exploiting the temporal locality inherent to spin summations. Combined, these optimizations result in speedups between 2.4x and 5.5x over the NCC quantum chemistry software package. In addition to such a performance boost, our algorithm can perform the spin summations in-place, thus reducing the memory footprint by 2x over an out-of-place variant.

cs.MS↗

arXiv2017

Introducing Geometric Algebra to Geometric Computing Software Developers: A Computational Thinking Approach

Designing software systems for Geometric Computing applications can be a challenging task. Software engineers typically use software abstractions to hide and manage the high complexity of such systems. Without the presence of a unifying algebraic system to describe geometric models, the use of software abstractions alone can result in many design and maintenance problems. Geometric Algebra (GA) can be a universal abstract algebraic language for software engineering geometric computing applications. Few sources, however, provide enough information about GA-based software implementations targeting the software engineering community. In particular, successfully introducing GA to software engineers requires quite different approaches from introducing GA to mathematicians or physicists. This article provides a high-level introduction to the abstract concepts and algebraic representations behind the elegant GA mathematical structure. The article focuses on the conceptual and representational abstraction levels behind GA mathematics with sufficient references for more details. In addition, the article strongly recommends applying the methods of Computational Thinking in both introducing GA to software engineers, and in using GA as a mathematical language for developing Geometric Computing software systems.

cs.MS↗

arXiv2017

Sparse Matrix Multiplication On An Associative Processor

Sparse matrix multiplication is an important component of linear algebra computations. Implementing sparse matrix multiplication on an associative processor (AP) enables high level of parallelism, where a row of one matrix is multiplied in parallel with the entire second matrix, and where the execution time of vector dot product does not depend on the vector size. Four sparse matrix multiplication algorithms are explored in this paper, combining AP and baseline CPU processing to various levels. They are evaluated by simulation on a large set of sparse matrices. The computational complexity of sparse matrix multiplication on AP is shown to be an O(nnz) where nnz is the number of nonzero elements. The AP is found to be especially efficient in binary sparse matrix multiplication. AP outperforms conventional solutions in power efficiency.

cs.MS↗

arXiv2017

Parallel Matrix-Free Implementation of Frequency-Domain Finite Difference Methods for Cluster Computing

Full-wave 3D electromagnetic simulations of complex planar devices, multilayer interconnects, and chip packages are presented for wide-band frequency-domain analysis using the finite difference integration technique developed in the PETSc software package. Initial reordering of the index assignment to the unknowns makes the resulting system matrix diagonally dominant. The rearrangement also facilitates the decomposition of large domain into slices for passing the mesh information to different machines. Matrix-free methods are then exploited to minimize the number of element-wise multiplications and memory requirements in the construction of the system of linear equations. Besides, the recipes provide extreme ease of modifications in the kernel of the code. The applicability of different Krylov subspace solvers is investigated. The accuracy is checked through comparisons with CST MICROWAVE STUDIO transient solver results. The parallel execution of the compiled code on specific number of processors in multi-core distributed-memory architectures demonstrate high scalability of the computational algorithm.

cs.CE↗

arXiv2017

Automatic Differentiation using Constraint Handling Rules in Prolog

Automatic differentiation is a technique which allows a programmer to define a numerical computation via compositions of a broad range of numeric and computational primitives and have the underlying system support the computation of partial derivatives of the result with respect to any of its inputs, without making any finite difference approximations, and without manipulating large symbolic expressions representing the computation. This note describes a novel approach to reverse mode automatic differentiation using constraint logic programmming, specifically, the constraint handling rules (CHR) library of SWI Prolog, resulting in a very small (50 lines of code) implementation. When applied to a differentiation-based implementation of the inside-outside algorithm for parameter learning in probabilistic grammars, the CHR based implementations outperformed two well-known frameworks for optimising differentiable functions, Theano and TensorFlow, by a large margin.

cs.MS↗

arXiv2017

A randomized Halton algorithm in R

Randomized quasi-Monte Carlo (RQMC) sampling can bring orders of magnitude reduction in variance compared to plain Monte Carlo (MC) sampling. The extent of the efficiency gain varies from problem to problem and can be hard to predict. This article presents an R function rhalton that produces scrambled versions of Halton sequences. On some problems it brings efficiency gains of several thousand fold. On other problems, the efficiency gain is minor. The code is designed to make it easy to determine whether a given integrand will benefit from RQMC sampling. An RQMC sample of n points in $[0,1]^d$ can be extended later to a larger n and/or d.

stat.CO↗

arXiv2018

Parareal Algorithm Implementation and Simulation in Julia

We present a full implementation of the parareal algorithm---an integration technique to solve differential equations in parallel---in the Julia programming language for a fully general, first-order, initial-value problem. We provide a brief overview of Julia---a concurrent programming language for scientific computing. Our implementation of the parareal algorithm accepts both coarse and fine integrators as functional arguments. We use Euler's method and another Runge-Kutta integration technique as the integrators in our experiments. We also present a simulation of the algorithm for purposes of pedagogy and as a tool for investigating the performance of the algorithm.

cs.MS↗

arXiv2017

Geometry-Oblivious FMM for Compressing Dense SPD Matrices

We present GOFMM (geometry-oblivious FMM), a novel method that creates a hierarchical low-rank approximation, "compression," of an arbitrary dense symmetric positive definite (SPD) matrix. For many applications, GOFMM enables an approximate matrix-vector multiplication in $N \log N$ or even $N$ time, where $N$ is the matrix size. Compression requires $N \log N$ storage and work. In general, our scheme belongs to the family of hierarchical matrix approximation methods. In particular, it generalizes the fast multipole method (FMM) to a purely algebraic setting by only requiring the ability to sample matrix entries. Neither geometric information (i.e., point coordinates) nor knowledge of how the matrix entries have been generated is required, thus the term "geometry-oblivious." Also, we introduce a shared-memory parallel scheme for hierarchical matrix computations that reduces synchronization barriers. We present results on the Intel Knights Landing and Haswell architectures, and on the NVIDIA Pascal architecture for a variety of matrices.

math.NA↗

arXiv2017

Compiling LATEX to computer algebra-enabled HTML5

This document explains how to create or modify an existing LATEX document with commands enabling computations in the HTML5 output: when the reader opens the HTML5 output, he can run a computation in his browser, or modify the command to be executed and run it. This is done by combining different softwares: hevea for compilation to HTML5, giac.js for the CAS computing kernel (itself compiled from the C++ Giac library with emscripten), and a modified version of itex2MML for fast and nice rendering in MathML in browsers that support MathML.

cs.SC↗

arXiv2017

Adaptive Modular Exponentiation Methods v.s. Python's Power Function

In this paper we use Python to implement two efficient modular exponentiation methods: the adaptive m-ary method and the adaptive sliding-window method of window size k, where both m's are adaptively chosen based on the length of exponent. We also conduct the benchmark for both methods. Evaluation results show that compared to the industry-standard efficient implementations of modular power function in CPython and Pypy, our algorithms can reduce 1-5% computing time for exponents with more than 3072 bits.

cs.DS↗

arXiv2017

Applying the Polyhedral Model to Tile Time Loops in Devito

The run time of many scientific computation applications for numerical methods is heavily dependent on just a few multi-dimensional loop nests. Since these applications are often limited by memory bandwidth rather than computational resources they can benefit greatly from any optimizations which decrease the run time of their loops by improving data reuse and thus reducing the total memory traffic. Some of the most effective of these optimizations are not suitable for development by hand or require advanced software engineering knowledge which is beyond the level of many researchers who are not specialists in code optimization. Several tools exist to automate the generation of high-performance code for numerical methods, such as Devito which produces code for finite-difference approximations typically used in the seismic imaging domain. We present a loop-tiling optimization which can be applied to Devito-generated loops and improves run time by up to 27.5%, and options for automating this optimization in the Devito framework.

cs.PL↗