SEARCH · arXiv Science
Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This paper describes generalized polylogarithms, multiple polylogarithms, and multiple zeta values, along with their implementation in Maple 2018. This set of related functions is of interest in high energy physics as well as in number theory. Algorithms for the analytical manipulation and numerical evaluation of these functions are described, along with the way these features are implemented in Maple.
In (Franceschi et al., 2018) we proposed a unified mathematical framework, grounded on bilevel programming, that encompasses gradient-based hyperparameter optimization and meta-learning. We formulated an approximate version of the problem where the inner objective is solved iteratively, and gave sufficient conditions ensuring convergence to the exact problem. In this work we show how to optimize learning rates, automatically weight the loss of single examples and learn hyper-representations with Far-HO, a software package based on the popular deep learning framework TensorFlow that allows to seamlessly tackle both HO and ML problems.
We describe algorithms to compute elliptic functions and their relatives (Jacobi theta functions, modular forms, elliptic integrals, and the arithmetic-geometric mean) numerically to arbitrary precision with rigorous error bounds for arbitrary complex variables. Implementations in ball arithmetic are available in the open source Arb library. We discuss the algorithms from a concrete implementation point of view, with focus on performance at tens to thousands of digits of precision.
The tensor-tensor product (t-product) [M. E. Kilmer and C. D. Martin, 2011] is a natural generalization of matrix multiplication. Based on t-product, many operations on matrix can be extended to tensor cases, including tensor SVD, tensor spectral norm, tensor nuclear norm [C. Lu, et al., 2018] and many others. The linear algebraic structure of tensors are similar to the matrix cases. We develop a Matlab toolbox to implement several basic operations on tensors based on t-product. The toolbox is available at https://github.com/canyilu/tproduct.
The CP tensor decomposition is a low-rank approximation of a tensor. We present a distributed-memory parallel algorithm and implementation of an alternating optimization method for computing a CP decomposition of dense tensor data that can enforce nonnegativity of the computed low-rank factors. The principal task is to parallelize the matricized-tensor times Khatri-Rao product (MTTKRP) bottleneck subcomputation. The algorithm is computation efficient, using dimension trees to avoid redundant computation across MTTKRPs within the alternating method. Our approach is also communication efficient, using a data distribution and parallel algorithm across a multidimensional processor grid that can be tuned to minimize communication. We benchmark our software on synthetic as well as hyperspectral image and neuroscience dynamic functional connectivity data, demonstrating that our algorithm scales well to 100s of nodes (up to 4096 cores) and is faster and more general than the currently available parallel software.
Finite-difference methods are widely used in solving partial differential equations. In a large problem set, approximations can take days or weeks to evaluate, yet the bulk of computation may occur within a single loop nest. The modelling process for researchers is not straightforward either, requiring models with differential equations to be translated into stencil kernels, then optimised separately. One tool that seeks to speed up and eliminate mistakes from this tedious procedure is Devito, used to efficiently employ finite-difference methods. In this work, we implement time-tiling, a loop nest optimisation, in Devito yielding a decrease in runtime of up to 45%, and at least 20% across stencils from the acoustic wave equation family, widely used in Devito's target domain of seismic imaging. We present an estimator for arithmetic intensity under time-tiling and a model to predict runtime improvements in stencil computations. We also consider generalisation of time-tiling to imperfect loop nests, a less widely studied problem.
The dune-functions Dune module provides interfaces for functions and function space bases. It forms one abstraction level above grids, shape functions, and linear algebra, and provides infrastructure for full discretization frameworks like dune-pdelab and dune-fem. This document describes the function space bases provided by dune-functions. These are based on an abstract description of bases for product spaces as trees of simpler bases. From this description, many different numberings of degrees of freedom by multi-indices can be derived in a natural way. We describe the abstract concepts, document the programmer interface, and give a complete example program that solves the stationary Stokes equation using Taylor-Hood elements.
This paper gives the first algorithm for finding a set of natural $ε$-clusters of complex zeros of a triangular system of polynomials within a given polybox in $\mathbb{C}^n$, for any given $ε>0$. Our algorithm is based on a recent near-optimal algorithm of Becker et al (2016) for clustering the complex roots of a univariate polynomial where the coefficients are represented by number oracles. Our algorithm is numeric, certified and based on subdivision. We implemented it and compared it with two well-known homotopy solvers on various triangular systems. Our solver always gives correct answers, is often faster than the homotopy solver that often gives correct answers, and sometimes faster than the one that gives sometimes correct results.
We describe Ccluster, a software for computing natural $ε$-clusters of complex roots in a given box of the complex plane. This algorithm from Becker et al.~(2016) is near-optimal when applied to the benchmark problem of isolating all complex roots of an integer polynomial. It is one of the first implementations of a near-optimal algorithm for complex roots. We describe some low level techniques for speeding up the algorithm. Its performance is compared with the well-known MPSolve library and Maple.
In this work, we consider the solution of boundary integral equations by means of a scalable hierarchical matrix approach on clusters equipped with graphics hardware, i.e. graphics processing units (GPUs). To this end, we extend our existing single-GPU hierarchical matrix library hmglib such that it is able to scale on many GPUs and such that it can be coupled to arbitrary application codes. Using a model GPU implementation of a boundary element method (BEM) solver, we are able to achieve more than 67 percent relative parallel speed-up going from 128 to 1024 GPUs for a model geometry test case with 1.5 million unknowns and a real-world geometry test case with almost 1.2 million unknowns. On 1024 GPUs of the cluster Titan, it takes less than 6 minutes to solve the 1.5 million unknowns problem, with 5.7 minutes for the setup phase and 20 seconds for the iterative solver. To the best of the authors' knowledge, we here discuss the first fully GPU-based distributed-memory parallel hierarchical matrix Open Source library using the traditional H-matrix format and adaptive cross approximation with an application to BEM problems.
This paper presents the implementation of a HLLC finite volume solver using GPU technology for the solution of shallow water problems in two dimensions. It compares both CPU and GPU approaches for implementing all the solver's steps. The technology of graphics and central processors is highlighted with a particular emphasis on the CUDA architecture of NVIDIA. The simple and well-documented Application Programming Interface (CUDA API) facilitates the use of the display card workstation as an additional computer unit to the central processor. Four professional solutions of the NVIDIA Quadro line are tested. Comparison tests between CPU and GPU are carried out on unstructured grids of small sizes (up to 10,000 elements), medium and large sizes (up to 10,000,000 elements). For all test cases, the accuracy of results is of the same order of magnitude for both approaches. Furthermore, the obtained speed gains with the GPU strongly depend on the model of the graphics card, the size of the problem and the simulation time.
Stencil computations are a key part of many high-performance computing applications, such as image processing, convolutional neural networks, and finite-difference solvers for partial differential equations. Devito is a framework capable of generating highly-optimized code given symbolic equations expressed in Python, specialized in, but not limited to, affine (stencil) codes. The lowering process---from mathematical equations down to C++ code---is performed by the Devito compiler through a series of intermediate representations. Several performance optimizations are introduced, including advanced common sub-expressions elimination, tiling and parallelization. Some of these are obtained through well-established stencil optimizers, integrated in the back-end of the Devito compiler. The architecture of the Devito compiler, as well as the performance optimizations that are applied when generating code, are presented. The effectiveness of such performance optimizations is demonstrated using operators drawn from seismic imaging applications.
In this paper we present the new Dune-Python module which provides Python bindings for the Dune core, which is a C++ environment for solving partial differential equations. The aim of this new module is to firstly provide the general infrastructure for exporting realizations of statically polymorphic interfaces based on just-in-time compilation and secondly to provide bindings for the central interfaces of the dune core modules. In the first release we focus on the grid interface. Our aim is to only introduce a thin layer when passing objects into Python which can be removed when the object is passed back into a C++ algorithm. Thus no efficiency is lost and little additional code maintenance cost is incurred. To make the transition for Dune users to the Python environment straightforward the Python classes provide a very similar interface to their C++ counterparts. In addition, vectorized versions of many interfaces allow for more efficient code on the Python side. The infrastructure for exporting these interfaces and the resulting bindings for a Dune grid are explained in detail in this paper for both experienced Dune users and others interested in a flexible Python environment for implementing grid based schemes for solving partial differential equations.
Performant numerical solving of differential equations is required for large-scale scientific modeling. In this manuscript we focus on two questions: (1) how can researchers empirically verify theoretical advances and consistently compare methods in production software settings and (2) how can users (scientific domain experts) keep up with the state-of-the-art methods to select those which are most appropriate? Here we describe how the confederated modular API of DifferentialEquations.jl addresses these concerns. We detail the package-free API which allows numerical methods researchers to readily utilize and benchmark any compatible method directly in full-scale scientific applications. In addition, we describe how the complexity of the method choices is abstracted via a polyalgorithm. We show how scientific tooling built on top of DifferentialEquations.jl, such as packages for dynamical systems quantification and quantum optics simulation, both benefit from this structure and provide themselves as convenient benchmarking tools.
The representation of units and dimensions in informatics systems is barely codified and often ignored. For instance, the major languages used in scientific computing (Fortran, C and Python), have no type for dimension or unit, and so physical quantities are represented in a program by variables of type real, resulting in the possibility of unit or dimensional errors. In view of this danger, many authors have proposed language schemes for unit-checking and conversion. However, since many physical quantities have the same units, it is possible for a block of code to be unit-compatible, but still physically meaningless. We demonstrate the limitations of three Python unit-libraries and present a justification and method for checking kind-of-quantity.
This is a tutorial in applied and computational topology and topological data analysis. It is illustrated with numerous computational examples that utilize Gudhi library. It is under constant development, so please do not consider this version as final.
Algorithmic differentiation (AD) allows exact computation of derivatives given only an implementation of an objective function. Although many AD tools are available, a proper and efficient implementation of AD methods is not straightforward. The existing tools are often too different to allow for a general test suite. In this paper, we compare fifteen ways of computing derivatives including eleven automatic differentiation tools implementing various methods and written in various languages (C++, F#, MATLAB, Julia and Python), two symbolic differentiation tools, finite differences, and hand-derived computation. We look at three objective functions from computer vision and machine learning. These objectives are for the most part simple, in the sense that no iterative loops are involved, and conditional statements are encapsulated in functions such as {\tt abs} or {\tt logsumexp}. However, it is important for the success of algorithmic differentiation that such `simple' objective functions are handled efficiently, as so many problems in computer vision and machine learning are of this form. Of course, our results depend on programmer skill, and familiarity with the tools. However, we contend that this paper presents an important datapoint: a skilled programmer devoting roughly a week to each tool produced the timings we present. We have made our implementations available as open source to allow the community to replicate and update these benchmarks.
Clawpack is a library for solving nonlinear hyperbolic partial differential equations using high-resolution finite volume methods based on Riemann solvers and limiters. It supports Adaptive Mesh Refinement (AMR), which is essential in solving multi-scale problems. Recently, we added capabilities to accelerate the code by using the Graphics Process Unit (GPU). Routines that manage CPU and GPU AMR data and facilitate the execution of GPU kernels are added. Customized and CPU thread-safe memory managers are designed to manage GPU and CPU memory pools, which is essential in eliminating the overhead of memory allocation and de-allocation. A global reduction is conducted every time step for dynamically adjusting the time step based on Courant number restrictions. Some small GPU kernels are merged into bigger kernels, which greatly reduces kernel launching overhead. A speed-up between $2$ and $3$ for the total running time is observed in an acoustics benchmark problem.