arXiv subjects

Jason Morton

Publications and source records attributed to Jason Morton.

At least 19 recordsLinked to original sources

arXiv2024

Verifiable evaluations of machine learning models using zkSNARKs

In a world of increasing closed-source commercial machine learning models, model evaluations from developers must be taken at face value. These benchmark results-whether over task accuracy, bias evaluations, or safety checks-are traditionally impossible to verify by a model end-user without the costly or impossible process of re-performing the benchmark on black-box model outputs. This work presents a method of verifiable model evaluation using model inference through zkSNARKs. The resulting zero-knowledge computational proofs of model outputs over datasets can be packaged into verifiable evaluation attestations showing that models with fixed private weights achieve stated performance or fairness metrics over public inputs. We present a flexible proving system that enables verifiable attestations to be performed on any standard neural network model with varying compute requirements. For the first time, we demonstrate this across a sample of real-world models and highlight key challenges and design solutions. This presents a new transparency paradigm in the verifiable evaluation of private models.

cs.LG↗

arXiv2017

Contextuality from missing and versioned data

Traditionally categorical data analysis (e.g. generalized linear models) works with simple, flat datasets akin to a single table in a database with no notion of missing data or conflicting versions. In contrast, modern data analysis must deal with distributed databases with many partial local tables that need not always agree. The computational agents tabulating these tables are spatially separated, with binding speed-of-light constraints and data arriving too rapidly for these distributed views ever to be fully informed and globally consistent. Contextuality is a mathematical property which describes a kind of inconsistency arising in quantum mechanics (e.g. in Bell's theorem). In this paper we show how contextuality can arise in common data collection scenarios, including missing data and versioning (as in low-latency distributed databases employing snapshot isolation). In the companion paper, we develop statistical models adapted to this regime.

cs.DB↗

arXiv2017

A Complete Set of Invariants for LU-Equivalence of Density Operators

We show that two density operators of mixed quantum states are in the same local unitary orbit if and only if they agree on polynomial invariants in a certain Noetherian ring for which degree bounds are known in the literature. This implicitly gives a finite complete set of invariants for local unitary equivalence. This is done by showing that local unitary equivalence of density operators is equivalent to local ${\rm GL}$ equivalence and then using techniques from algebraic geometry and geometric invariant theory. We also classify the SLOCC polynomial invariants and give a degree bound for generators of the invariant ring in the case of $n$-qubit pure states. Of course it is well known that polynomial invariants are not a complete set of invariants for SLOCC.

math.RT↗

arXiv2017

The Invariant Ring Of m Matrices Under The Adjoint Action By a Product Of General Linear Groups

Let $V=V_1 \otimes \cdots \otimes V_n$ be a vector space over an algebraically closed field $K$ of characteristic zero with $\dim(V_i)=d_i$. We study the ring of polynomial invariants $K[\operatorname{End}(V)^{\oplus m}]^{\operatorname{GL}_{\mathbf{d}}}$ of $m$ endomorphisms of $V$ under the adjoint action of $\operatorname{GL}_{\mathbf{d}}:=\operatorname{GL}(V_1) \times \cdots \times \operatorname{GL}(V_n)$. We find that the ring is generated by certain generalized trace monomials $\operatorname{Tr}^M_σ$ where $M$ is a multiset with entries in $[m]=\{1,\dots, m\}$ and $σ\in \mathcal{S}_m^n$ is a choice of $n$ permutations of $[m]$. We find that $K[\operatorname{End}(V)^{\oplus m}]^{\operatorname{GL}_{\mathbf{d}}}$ is generated by the $\operatorname{Tr}^M_σ$ of degree at most $\frac{3}{8}m\dim(V)^6$.

math.RT↗

arXiv2015

Dimension of Marginals of Kronecker Product Models

A Kronecker product model is the set of visible marginal probability distributions of an exponential family whose sufficient statistics matrix factorizes as a Kronecker product of two matrices, one for the visible variables and one for the hidden variables. We estimate the dimension of these models by the maximum rank of the Jacobian in the limit of large parameters. The limit is described by the tropical morphism; a piecewise linear map with pieces corresponding to slicings of the visible matrix by the normal fan of the hidden matrix. We obtain combinatorial conditions under which the model has the expected dimension, equal to the minimum of the number of natural parameters and the dimension of the ambient probability simplex. Additionally, we prove that the binary restricted Boltzmann machine always has the expected dimension.

stat.ML↗

arXiv2015

Computing the Tutte Polynomial of Lattice Path Matroids Using Determinantal Circuits

We give a quantum-inspired $O(n^4)$ algorithm computing the Tutte polynomial of a lattice path matroid, where $n$ is the size of the ground set of the matroid. Furthermore, this can be improved to $O(n^2)$ arithmetic operations if we evaluate the Tutte polynomial on a given input, fixing the values of the variables. The best existing algorithm, found in 2004, was $O(n^5)$, and the problem has only been known to be polynomial time since 2003. Conceptually, our algorithm embeds the computation in a determinant using a recently demonstrated equivalence of categories useful for counting problems such as those that appear in simulating quantum systems.

math.CO↗

arXiv2014

Belief propagation in monoidal categories

We discuss a categorical version of the celebrated belief propagation algorithm. This provides a way to prove that some algorithms which are known or suspected to be analogous, are actually identical when formulated generically. It also highlights the computational point of view in monoidal categories.

math.CT↗

arXiv2014

Tensor Network Contractions for #SAT

The computational cost of counting the number of solutions satisfying a Boolean formula, which is a problem instance of #SAT, has proven subtle to quantify. Even when finding individual satisfying solutions is computationally easy (e.g. 2-SAT, which is in P), determining the number of solutions is #P-hard. Recently, computational methods simulating quantum systems experienced advancements due to the development of tensor network algorithms and associated quantum physics-inspired techniques. By these methods, we give an algorithm using an axiomatic tensor contraction language for n-variable #SAT instances with complexity $O((g+cd)^{O(1)} 2^c)$ where $c$ is the number of COPY-tensors, $g$ is the number of gates, and $d$ is the maximal degree of any COPY-tensor. Thus, counting problems can be solved efficiently when their tensor network expression has at most $O(\log c)$ COPY-tensors and polynomial fan-out. This framework also admits an intuitive proof of a variant of the Tovey conjecture (the r,1-SAT instance of the Dubois-Tovey theorem). This study increases the theory, expressiveness and application of tensor based algorithmic tools and provides an alternative insight on these problems which have a long history in statistical physics and computer science.

quant-ph↗

arXiv2014

When Does a Mixture of Products Contain a Product of Mixtures?

We derive relations between theoretical properties of restricted Boltzmann machines (RBMs), popular machine learning models which form the building blocks of deep learning models, and several natural notions from discrete mathematics and convex geometry. We give implications and equivalences relating RBM-representable probability distributions, perfectly reconstructible inputs, Hamming modes, zonotopes and zonosets, point configurations in hyperplane arrangements, linear threshold codes, and multi-covering numbers of hypercubes. As a motivating application, we prove results on the relative representational power of mixtures of product distributions and products of mixtures of pairs of product distributions (RBMs) that formally justify widely held intuitions about distributed representations. In particular, we show that a mixture of products requiring an exponentially larger number of parameters is needed to represent the probability distributions which can be obtained as products of mixtures.

stat.ML↗

arXiv2014

Algebraic Geometry of Matrix Product States

We quantify the representational power of matrix product states (MPS) for entangled qubit systems by giving polynomial expressions in a pure quantum state's amplitudes which hold if and only if the state is a translation invariant matrix product state or a limit of such states. For systems with few qubits, we give these equations explicitly, considering both periodic and open boundary conditions. Using the classical theory of trace varieties and trace algebras, we explain the relationship between MPS and hidden Markov models and exploit this relationship to derive useful parameterizations of MPS. We make four conjectures on the identifiability of MPS parameters.

quant-ph↗

arXiv2014

Generalized Counting Constraint Satisfaction Problems With Determinantal Circuits

Generalized counting constraint satisfaction problems include Holant problems with planarity restrictions; polynomial-time algorithms for such problems include matchgates and matchcircuits, which are based on Pfaffians. In particular, they use gates which are expressible in terms of a vector of sub-Pfaffians of a skew-symmetric matrix. We introduce a new type of circuit based instead on determinants, with seemingly different expressive power. In these determinantal circuits, a gate is represented by the vector of all minors of an arbitrary matrix. Determinantal circuits permit a different class of gates. Applications of these circuits include proofs of theorems from algebraic graph theory including the Chung-Langlands formula for the number of rooted spanning forests of a graph and computing Tutte Polynomials of certain matroids. They also give a strategy for simulating quantum circuits with closed timelike curves. Monoidal category theory provides a useful language for discussing such counting problems, turning combinatorial restrictions into categorical properties. We introduce the counting problem in monoidal categories and count-preserving functors as a way to study FP subclasses of problems in settings which are generally #P-hard. Using this machinery we show that, surprisingly, determinantal circuits can be simulated by Pfaffian circuits at quadratic cost.

math.CT↗

arXiv2014

Discrete Restricted Boltzmann Machines

We describe discrete restricted Boltzmann machines: probabilistic graphical models with bipartite interactions between visible and hidden discrete variables. Examples are binary restricted Boltzmann machines and discrete naive Bayes models. We detail the inference functions and distributed representations arising in these models in terms of configurations of projected products of simplices and normal fans of products of simplices. We bound the number of hidden variables, depending on the cardinalities of their state spaces, for which these models can approximate any probability distribution on their visible states to any given accuracy. In addition, we use algebraic methods and coding theory to compute their dimension.

stat.ML↗

arXiv2012

Kernels and Submodels of Deep Belief Networks

We study the mixtures of factorizing probability distributions represented as visible marginal distributions in stochastic layered networks. We take the perspective of kernel transitions of distributions, which gives a unified picture of distributed representations arising from Deep Belief Networks (DBN) and other networks without lateral connections. We describe combinatorial and geometric properties of the set of kernels and products of kernels realizable by DBNs as the network parameters vary. We describe explicit classes of probability distributions, including exponential families, that can be learned by DBNs. We use these submodels to bound the maximal and the expected Kullback-Leibler approximation errors of DBNs from above depending on the number of hidden layers and units that they contain.

stat.ML↗

arXiv2012

Undecidability in Tensor Network States

Recent work has examined how undecidable problems can arise in quantum information science. We augment this by introducing three new undecidable problems stated in terms of tensor networks. These relate to ideas of Penrose about the physicality of a spin-network representing a physical process, closed timelike curves, and Boolean relation theory. Seemingly slight modifications of the constraints on the topology or the tensor families generating the networks leads to problems that transition from decidable, to undecidable to even always satisfiable.

quant-ph↗

arXiv2010

Pfaffian Circuits

It remains an open question whether the apparent additional power of quantum computation derives inherently from quantum mechanics, or merely from the flexibility obtained by "lifting" Boolean functions to linear operators and evaluating their composition cleverly. Holographic algorithms provide a useful avenue for exploring this question. We describe a new, simplified construction of holographic algorithms in terms of Pfaffian circuits. Novel proofs of some key results are provided, and we extend the approach of [34] to nonsymmetric, odd, and homogenized signatures, circuits, and various models of execution flow. This shows our approach is as powerful as the matchgate approach. Holographic algorithms provide in general $O(n^{ω_p})$ time algorithms, where $ω_p$ is the order of Pfaffian evaluation in the ring of interest (with $1.19 \leq ω_p \leq 3$ depending on the ring) and $n$ is the number of inclusions of variables into clauses. Our approach often requires just the evaluation of an $n \times n$ Pfaffian, and at most needs an additional two rows per gate, whereas the matchgate approach is quartic in the arity of the largest gate. We give examples (even before any change of basis) including efficient algorithms for certain lattice path problems and an $O(n^{ω_p})$ algorithm for evaluation of Tutte polynomials of lattice path matroids. Finally we comment on some of the geometric considerations in analyzing Pfaffian circuits under arbitrary basis change. Connections are made to the sum-product algorithm, classical simulation of quantum computation, and SLOCC equivalent entangled states.

math.CO↗

arXiv2009

Geometry of the restricted Boltzmann machine

The restricted Boltzmann machine is a graphical model for binary random variables. Based on a complete bipartite graph separating hidden and observed variables, it is the binary analog to the factor analysis model. We study this graphical model from the perspectives of algebraic statistics and tropical geometry, starting with the observation that its Zariski closure is a Hadamard power of the first secant variety of the Segre variety of projective lines. We derive a dimension formula for the tropicalized model, and we use it to show that the restricted Boltzmann machine is identifiable in many cases. Our methods include coding theory and geometry of linear threshold functions.

stat.ML↗

arXiv2009

Holographic algorithms without matchgates

The theory of holographic algorithms, which are polynomial time algorithms for certain combinatorial counting problems, yields insight into the hierarchy of complexity classes. In particular, the theory produces algebraic tests for a problem to be in the class P. In this article we streamline the implementation of holographic algorithms by eliminating one of the steps in the construction procedure, and generalize their applicability to new signatures. Instead of matchgates, which are weighted graph fragments that replace vertices of a natural bipartite graph G associated to a problem P, our approach uses only only a natural number-of-edges by number-of-edges matrix associated to G. An easy-to-compute multiple of its Pfaffian is the number of solutions to the counting problem. This simplification improves our understanding of the applicability of holographic algorithms, indicates a more geometric approach to complexity classes, and facilitates practical implementations. The generalized applicability arises because our approach allows for new algebraic tests that are different from the "Grassmann-Plucker identities" used up until now. Natural problems treatable by these new methods have been previously considered in a different context, and we present one such example.

cs.CC↗

arXiv2008

Graphical models for correlated defaults

A simple graphical model for correlated defaults is proposed, with explicit formulas for the loss distribution. Algebraic geometry techniques are employed to show that this model is well posed for default dependence: it represents any given marginal distribution for single firms and pairwise correlation matrix. These techniques also provide a calibration algorithm based on maximum likelihood estimation. Finally, the model is compared with standard normal copula model in terms of tails of the loss distribution and implied correlation smile.

q-fin.CP↗