arXiv subjects

Richard Samuelson

Publications and source records attributed to Richard Samuelson.

5 recordsLinked to original sources

arXiv2025

DiagrammaticLearning: A Graphical Language for Compositional Training Regimes

Motivated by deep learning regimes with multiple interacting yet distinct model components, we introduce learning diagrams, graphical depictions of training setups that capture parameterized learning as data rather than code. A learning diagram compiles to a unique loss function on which component models are trained. The result of training on this loss is a collection of models whose predictions ``agree" with one another. We show that a number of popular learning setups such as few-shot multi-task learning, knowledge distillation, and multi-modal learning can be depicted as learning diagrams. We further implement learning diagrams in a library that allows users to build diagrams of PyTorch and Flux.jl models. By implementing some classic machine learning use cases, we demonstrate how learning diagrams allow practitioners to build complicated models as compositions of smaller components, identify relationships between workflows, and manipulate models during or after training. Leveraging a category theoretic framework, we introduce a rigorous semantics for learning diagrams that puts such operations on a firm mathematical foundation.

cs.LG↗

arXiv2024

A Categorical Treatment of Open Linear Systems

An open stochastic system \`a la Jan Willems is a system affected by two qualitatively different kinds of uncertainty: one is probabilistic fluctuation, and the other one is nondeterminism caused by a fundamental lack of information. We present a formalization of open stochastic systems in the language of category theory. Central to this is the notion of copartiality which models how the lack of information propagates through a system (corresponding to the coarseness of sigma-algebras in Willems' work). As a concrete example, we study extended Gaussian distributions, which combine Gaussian probability with nondeterminism and correspond precisely to Willems' notion of Gaussian linear systems. We describe them both as measure-theoretic and abstract categorical entities, which enables us to rigorously describe a variety of phenomena like noisy physical laws and uninformative priors in Bayesian statistics. The category of extended Gaussian maps can be seen as a mutual generalization of Gaussian probability and linear relations, which connects the literature on categorical probability with ideas from control theory like signal-flow diagrams.

cs.LO↗

arXiv2024

Graphical Quadratic Algebra

Convex analysis and Gaussian probability are tightly connected, as mostly evident in the theory of linear regression. Our work introduces an algebraic perspective on such relationship, in the form of a diagrammatic calculus of string diagrams, called Graphical Quadratic Algebra (GQA). We show that GQA is a complete axiomatisation for the category of quadratic relations, a compositional formulation of quadratic problems. Moreover, we identify a sub-theory of GQA which is complete for the category of Gaussian probabilistic processes. We show how GQA may be used to study linear regression and probabilistic programming.

cs.LO↗

arXiv2023

Towards a Compositional Framework for Convex Analysis (with Applications to Probability Theory)

We introduce a compositional framework for convex analysis based on the notion of convex bifunction of Rockafellar. This framework is well-suited to graphical reasoning, and exhibits rich dualities such as the Legendre-Fenchel transform, while generalizing formalisms like graphical linear algebra, convex relations and convex programming. We connect our framework to probability theory by interpreting the Laplace approximation in its context: The exactness of this approximation on normal distributions means that logdensity is a functor from Gaussian probability (densities and integration) to concave bifunctions and maximization.

math.CT↗

arXiv2022

A Category for unifying Gaussian Probability and Nondeterminism

We introduce categories of extended Gaussian maps and Gaussian relations which unify Gaussian probability distributions with relational nondeterminism in the form of linear relations. Both have crucial and well-understood applications in statistics, engineering, and control theory, but combining them in a single formalism is challenging. It enables us to rigorously describe a variety of phenomena like noisy physical laws, Willems' theory of open systems and uninformative priors in Bayesian statistics. The core idea is to formally admit vector subspaces $D \subseteq X$ as generalized uniform probability distribution. Our formalism represents a first bridge between the literature on categorical systems theory (signal-flow diagrams, linear relations, hypergraph categories) and notions of probability theory.

cs.LO↗