SEARCH · arXiv Science

Results for “cs.MS”

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46Linked to original sources

arXiv2019

Solving Polynomial Systems with phcpy

The solutions of a system of polynomials in several variables are often needed, e.g.: in the design of mechanical systems, and in phase-space analyses of nonlinear biological dynamics. Reliable, accurate, and comprehensive numerical solutions are available through PHCpack, a FOSS package for solving polynomial systems with homotopy continuation. This paper explores new developments in phcpy, a scripting interface for PHCpack, over the past five years. For instance, phcpy is now available online through a JupyterHub server featuring Python2, Python3, and SageMath kernels. As small systems are solved in real-time by phcpy, they are suitable for interactive exploration through the notebook interface. Meanwhile, phcpy supports GPU parallelization, improving the speed and quality of solutions to much larger polynomial systems. From various model design and analysis problems in STEM, certain classes of polynomial system frequently arise, to which phcpy is well-suited.

cs.MS↗

arXiv2019

Algorithms and data structures for matrix-free finite element operators with MPI-parallel sparse multi-vectors

Traditional solution approaches for problems in quantum mechanics scale as $\mathcal O(M^3)$, where $M$ is the number of electrons. Various methods have been proposed to address this issue and obtain linear scaling $\mathcal O(M)$. One promising formulation is the direct minimization of energy. Such methods take advantage of physical localization of the solution, namely that the solution can be sought in terms of non-orthogonal orbitals with local support. In this work a numerically efficient implementation of sparse parallel vectors within the open-source finite element library deal.II is proposed. The main algorithmic ingredient is the matrix-free evaluation of the Hamiltonian operator by cell-wise quadrature. Based on an a-priori chosen support for each vector we develop algorithms and data structures to perform (i) matrix-free sparse matrix multivector products (SpMM), (ii) the projection of an operator onto a sparse sub-space (inner products), and (iii) post-multiplication of a sparse multivector with a square matrix. The node-level performance is analyzed using a roofline model. Our matrix-free implementation of finite element operators with sparse multivectors achieves the performance of 157 GFlop/s on Intel Cascade Lake architecture. Strong and weak scaling results are reported for a typical benchmark problem using quadratic and quartic finite element bases.

cs.MS↗

arXiv2020

GPU-based Parallel Computation Support for Stan

This paper details an extensible OpenCL framework that allows Stan to utilize heterogeneous compute devices. It includes GPU-optimized routines for the Cholesky decomposition, its derivative, other matrix algebra primitives and some commonly used likelihoods, with more additions planned for the near future. Stan users can now benefit from large speedups offered by GPUs with little effort and without changes to their existing Stan code. We demonstrate the practical utility of our work with two examples - logistic regression and Gaussian Process regression.

cs.MS↗

arXiv2019

bayes4psy -- an Open Source R Package for Bayesian Statistics in Psychology

Research in psychology generates interesting data sets and unique statistical modelling tasks. However, these tasks, while important, are often very specific, so appropriate statistical models and methods cannot be found in accessible Bayesian tools. As a result, the use of Bayesian methods is limited to those that have the technical and statistical fundamentals that are required for probabilistic programming. Such knowledge is not part of the typical psychology curriculum and is a difficult obstacle for psychology students and researchers to overcome. The goal of the bayes4psy package is to bridge this gap and offer a collection of models and methods to be used for data analysis that arises from psychology experiments and as a teaching tool for Bayesian statistics in psychology. The package contains Bayesian t-test and bootstrapping and models for analyzing reaction times, success rates, and colors. It also provides all the diagnostic, analytic and visualization tools for the modern Bayesian data analysis workflow.

stat.AP↗

arXiv2024

hyppo: A Multivariate Hypothesis Testing Python Package

We introduce hyppo, a unified library for performing multivariate hypothesis testing, including independence, two-sample, and k-sample testing. While many multivariate independence tests have R packages available, the interfaces are inconsistent and most are not available in Python. hyppo includes many state of the art multivariate testing procedures. The package is easy-to-use and is flexible enough to enable future extensions. The documentation and all releases are available at https://hyppo.neurodata.io.

stat.CO↗

arXiv2019

Multi-dimensional interpolations in C++

A C++ software design is presented that can be used to interpolate data in any number of dimensions. The design is based on a combination of templates of functional collections of elements and so-called type lists. The design allows for different search methodologies and interpolation techniques in each dimension. It is also possible to expand and reduce the number of dimensions, to interpolate composite data types and to produce on-the-fly additional values such as derivatives of the interpolating function.

cs.MS↗

arXiv2021

Unveiling patterns in xorshift128+ pseudorandom number generators

Xorshift128+ is a newly proposed pseudorandom number generator (PRNG), which is now the standard PRNG on a number of platforms. We demonstrate that three-dimensional plots of the random points generated by the generator have visible structures: they concentrate on particular planes in the cube. We provide a mathematical analysis of this phenomenon.

cs.MS↗

arXiv2019

Out-of-core singular value decomposition

Singular value decomposition (SVD) is a standard matrix factorization technique that produces optimal low-rank approximations of matrices. It has diverse applications, including machine learning, data science and signal processing. However, many common problems involve very large matrices that cannot fit in the main memory of commodity computers, making it impractical to use standard SVD algorithms that assume fast random access or large amounts of space for intermediate calculations. To address this issue, we have implemented an out-of-core (external memory) randomized SVD solution that is fully scalable and efficiently parallelizable. This solution factors both dense and sparse matrices of arbitrarily large size within arbitrarily small memory limits, efficiently using out-of-core storage as needed. It uses an innovative technique for partitioning matrices that lends itself to out-of-core and parallel processing, as well as memory and I/O use planning, automatic load balancing, performance tuning, and makes possible a number of other practical enhancements to the current state-of-the-art. Furthermore, by using persistent external storage (generally HDDs or SSDs), users can resume interrupted operations without having to recalculate previously performed steps, solving a major practical problem in factoring very large matrices.

cs.MS↗

arXiv2019

A Hermite-like basis for faster matrix-free evaluation of interior penalty discontinuous Galerkin operators

This work proposes a basis for improved throughput of matrix-free evaluation of discontinuous Galerkin symmetric interior penalty discretizations on hexahedral elements. The basis relies on ideas of Hermite polynomials. It is used in a fully discontinuous setting not for higher order continuity but to minimize the effective stencil width, namely to limit the neighbor access of an element to one data point for the function value and one for the derivative. The basis is extended to higher orders with nodal contributions derived from roots of Jacobi polynomials and extended to multiple dimensions with tensor products, which enable the use of sum factorization. The beneficial effect of the reduced data access on modern processors is shown. Furthermore, the viability of the basis in the context of multigrid solvers is analyzed. While a plain point-Jacobi approach is less efficient than with the best nodal polynomials, a basis change via sum-factorization techniques enables the combination of the fast matrix-vector products with effective multigrid constituents. The basis change is essentially for free on modern hardware because these computations can be hidden behind the cost of the data access.

math.NA↗

arXiv2019

PyLops -- A Linear-Operator Python Library for large scale optimization

Linear operators and optimisation are at the core of many algorithms used in signal and image processing, remote sensing, and inverse problems. For small to medium-scale problems, existing software packages (e.g., MATLAB, Python numpy and scipy) allow for explicitly building dense (or sparse) matrices and performing algebraic operations (e.g., computation of matrix-vector products and manipulation of matrices) with syntax that closely represents their corresponding analytical forms. However, many real application, large-scale operators do not lend themselves to explicit matrix representations, usually forcing practitioners to forego of the convenient linear-algebra syntax available for their explicit-matrix counterparts. PyLops is an open-source Python library providing a flexible and scalable framework for the creation and combination of so-called linear operators, class-based entities that represent matrices and inherit their associated syntax convenience, but do not rely on the creation of explicit matrices. We show that PyLops operators can dramatically reduce the memory load and CPU computations compared to explicit-matrix calculations, while still allowing users to seamlessly use their existing knowledge of compact matrix-based syntax that scales to any problem size because no explicit matrices are required.

cs.MS↗

arXiv2019

Testing performance with and without Block Low Rank Compression in MUMPS and the new PaStiX 6.0 for JOREK nonlinear MHD simulations

The interface to the MUMPS solver was updated in the JOREK MHD code to support Block Low Rank (BLR) compression and an interface to the new PaStiX solver version 6 has been implemented supporting BLR as well. First tests were carried out with JOREK, which solves a large sparse matrix system iteratively in each time step. For the preconditioning, a direct solver is applied in the code to sub-matrices, and at this point BLR was applied with the results being summarized in this report. For a simple case with a linearly growing mode, results with both solvers look promising with a considerable reduction of the memory consumption by several ten percent was obtained. A direct increase in performance was seen in particular configurations already. The choice of the BLR accuracy parameter $ε$ proves to be critical in this simple test and also in more realistic simulations, which were carried out only with MUMPS due to the limited time available. The more realistic test showed an increase in run time when using BLR, which was mitigated when using larger values of $ε$. However, the GMRes iterative solver does not reach convergence anymore when $ε$ is too large, since the preconditioner becomes too inaccurate in that case. It is thus critical to use an $ε$ as large as possible, while still reaching convergence. More tests regarding this optimum will be necessary in the future. BLR can also lead to an indirect speed-up in particular cases, when the simulation can be run on a smaller number of compute nodes due to the reduced memory consumption.

cs.PF↗

arXiv2019

pySOT and POAP: An event-driven asynchronous framework for surrogate optimization

This paper describes Plumbing for Optimization with Asynchronous Parallelism (POAP) and the Python Surrogate Optimization Toolbox (pySOT). POAP is an event-driven framework for building and combining asynchronous optimization strategies, designed for global optimization of expensive functions where concurrent function evaluations are useful. POAP consists of three components: a worker pool capable of function evaluations, strategies to propose evaluations or other actions, and a controller that mediates the interaction between the workers and strategies. pySOT is a collection of synchronous and asynchronous surrogate optimization strategies, implemented in the POAP framework. We support the stochastic RBF method by Regis and Shoemaker along with various extensions of this method, and a general surrogate optimization strategy that covers most Bayesian optimization methods. We have implemented many different surrogate models, experimental designs, acquisition functions, and a large set of test problems. We make an extensive comparison between synchronous and asynchronous parallelism and find that the advantage of asynchronous computation increases as the variance of the evaluation time or number of processors increases. We observe a close to linear speed-up with 4, 8, and 16 processors in both the synchronous and asynchronous setting.

math.OC↗

arXiv2021

GraphBLAST: A High-Performance Linear Algebra-based Graph Framework on the GPU

High-performance implementations of graph algorithms are challenging to implement on new parallel hardware such as GPUs because of three challenges: (1) the difficulty of coming up with graph building blocks, (2) load imbalance on parallel hardware, and (3) graph problems having low arithmetic intensity. To address some of these challenges, GraphBLAS is an innovative, on-going effort by the graph analytics community to propose building blocks based on sparse linear algebra, which will allow graph algorithms to be expressed in a performant, succinct, composable and portable manner. In this paper, we examine the performance challenges of a linear-algebra-based approach to building graph frameworks and describe new design principles for overcoming these bottlenecks. Among the new design principles is exploiting input sparsity, which allows users to write graph algorithms without specifying push and pull direction. Exploiting output sparsity allows users to tell the backend which values of the output in a single vectorized computation they do not want computed. Load-balancing is an important feature for balancing work amongst parallel workers. We describe the important load-balancing features for handling graphs with different characteristics. The design principles described in this paper have been implemented in "GraphBLAST", the first high-performance linear algebra-based graph framework on NVIDIA GPUs that is open-source. The results show that on a single GPU, GraphBLAST has on average at least an order of magnitude speedup over previous GraphBLAS implementations SuiteSparse and GBTL, comparable performance to the fastest GPU hardwired primitives and shared-memory graph frameworks Ligra and Gunrock, and better performance than any other GPU graph framework, while offering a simpler and more concise programming model.

cs.DC↗

arXiv2020

Domain-Driven Solver (DDS) Version 2.0: a MATLAB-based Software Package for Convex Optimization Problems in Domain-Driven Form

Domain-Driven Solver (DDS) is a MATLAB-based software package for convex optimization problems in Domain-Driven form [Karimi and Tunçel, arXiv:1804.06925]. The current version of DDS accepts every combination of the following function/set constraints: (1) symmetric cones (LP, SOCP, and SDP); (2) quadratic constraints that are SOCP representable; (3) direct sums of an arbitrary collection of 2-dimensional convex sets defined as the epigraphs of univariate convex functions (including as special cases geometric programming and entropy programming); (4) generalized power cone; (5) epigraphs of matrix norms (including as a special case minimization of nuclear norm over a linear subspace); (6) vector relative entropy; (7) epigraphs of quantum entropy and quantum relative entropy; and (8) constraints involving hyperbolic polynomials. DDS is a practical implementation of the infeasible-start primal-dual algorithm designed and analyzed in [Karimi and Tunçel, arXiv:1804.06925]. This manuscript contains the users' guide, as well as theoretical results needed for the implementation of the algorithms. To help the users, we included many examples. We also discussed some implementation details and techniques we used to improve the efficiency and further expansion of the software to cover the emerging classes of convex optimization problems.

math.OC↗

arXiv2020

SODECL: An Open Source Library for Calculating Multiple Orbits of a System of Stochastic Differential Equations in Parallel

Stochastic differential equations (SDEs) are widely used to model systems affected by random processes. In general, the analysis of an SDE model requires numerical solutions to be generated many times over multiple parameter combinations. However, this process often requires considerable computational resources to be practicable. Due to the embarrassingly parallel nature of the task, devices such as multi-core processors and graphics processing units (GPUs) can be employed for acceleration. Here, we present {\bf SODECL} (\url{https://github.com/avramidis/sodecl}), a software library that utilises such devices to calculate multiple orbits of an SDE model. To evaluate the acceleration provided by SODECL, we compared the time required to calculate multiple orbits of an exemplar stochastic model when one CPU core is used, to the time required when using all CPU cores or a GPU. In addition, to assess scalability, we investigated how the model size affected execution time on different parallel compute devices. Our results show that when using all 32 CPU cores of a high-end high-performance computing node, the task is accelerated by a factor of up to $\simeq$6.7, compared to when using a single CPU core. Executing the task on a high-end GPU yielded accelerations of up to $\simeq$4.5, compared to a single CPU core.

cs.MS↗

arXiv2019

Open Traffic Models -- A framework for hybrid simulation of transportation networks

This paper introduces a new approach to hybrid traffic modeling, along with its implementation in software. The software allows modelers to assign traffic models to individual links in a network. Each model implements a series of methods, refered to as the modeling interface. These methods are used by the program to exchange information between adjacent models. Traffic controllers are implemented in a similar manner. The paper outlines the important components of the method: the network description, the description of demands, and the modeling and control interfaces. We include tests demonstrating the propagation of congestion between pairs of macroscpoic, mesoscopic, and microscopic models. Open Traffic Models is an open source implementation of these concepts, and is available at https://github.com/ggomes/otm-sim.

cs.MS↗

arXiv2019

OpenMP parallelization of multiple precision Taylor series method

OpenMP parallelization of multiple precision Taylor series method is proposed. A very good parallel performance scalability and parallel efficiency inside one computation node of a CPU-cluster is observed. We explain the details of the parallelization on the classical example of the Lorentz equations. The same approach can be applied straightforwardly to a large class of chaotic dynamical systems.

cs.MS↗

arXiv2019

High Performance Block Incomplete LU Factorization

Many application problems that lead to solving linear systems make use of preconditioned Krylov subspace solvers to compute their solution. Among the most popular preconditioning approaches are incomplete factorization methods either as single-level approaches or within a multilevel framework. We will present a block incomplete factorization that is based on skillfully blocking the system initially and throughout the factorization. This approach allows for the use of cache-optimized dense matrix kernels such as level-3 BLAS or LAPACK. We will demonstrate how this block approach outperforms the scalar method often by orders of magnitude on modern architectures, paving the way for its prospective use inside various multilevel incomplete factorization approaches or other applications where the core part relies on an incomplete factorization.

math.NA↗