Gaussian Variant of Freivalds' Algorithm for Efficient and Reliable Matrix Product Verification

Abstract

In this article, we consider the general problem of checking the correctness of matrix multiplication. Given three $n \times n$ matrices $A$, $B$, and $C$, the goal is to verify that $A \times B=C$ without carrying out the computationally costly operations of matrix multiplication and comparing the product $A \times B$ with $C$, term by term. This is especially important when some or all of these matrices are very large, and when the computing environment is prone to soft errors. Here we extend Freivalds' algorithm to a Gaussian Variant of Freivalds' Algorithm (GVFA) by projecting the product $A \times B$ as well as $C$ onto a Gaussian random vector and then comparing the resulting vectors. The computational complexity of GVFA is consistent with that of Freivalds' algorithm, which is $O(n^{2})$. However, unlike Freivalds' algorithm, whose probability of a false positive is $2^{-k}$, where $k$ is the number of iterations. Our theoretical analysis shows that when $A \times B \neq C$, GVFA produces a false positive on set of inputs of measure zero with exact arithmetic. When we introduce round-off error and floating point arithmetic into our analysis, we can show that the larger this error, the higher the probability that GVFA avoids false positives. Moreover, by iterating GVFA $k$ times, the probability of a false positive decreases as $p^k$, where $p$ is a very small value depending on the nature of the fault on the result matrix and the arithmetic system's floating-point precision. Unlike deterministic algorithms, there do not exist any fault patterns that are completely undetectable with GVFA. Thus GVFA can be used to provide efficient fault tolerance in numerical linear algebra, and it can be efficiently implemented on modern computing architectures. In particular, GVFA can be very efficiently implemented on architectures with hardware support for fused multiply-add operations.