On the Sequentially Semiseparable Structure of SDE-Induced Kernels and Efficient Algorithms for Kernel-Based Estimation Problems

Abstract

We study kernel functions induced by a linear time-invariant (LTI) stochastic differential equation (SDE), an approach that embeds prior knowledge about the function to be estimated directly into the state-space description of the SDE. This kernel-design perspective is relevant across the many settings where kernels play a central role, including kernel methods in statistics, Gaussian process (GP) regression in machine learning, and kernel-based regularization methods in system identification. The resulting class of SDE-induced kernels includes several kernels of practical interest, such as Matérn kernels with half-integer smoothness, spline kernels, and related kernels used in system identification. In contrast to existing linear-time GP regression methods for these kernels, which are typically based on Kalman filtering and Rauch--Tung--Striebel smoothing on the latent state, we work directly with the SSS representation of the observed covariance matrix. This yields linear-complexity recursive algorithms for the matrix operations required in GP regression, including factorization, linear solves, log determinants, posterior covariance, log-marginal-likelihood gradients, and posterior sampling; the same generator-level recursions extend, with only minor modifications, to the analogous computations in kernel-based regularization methods for system identification. Compared with previous uses of SSS representations in GP regression, we derive efficient recursive algorithms for analytic marginal-likelihood gradients in three application-relevant cases, and we develop new recursions for posterior computations and Matheron-type sampling. Numerical experiments verify the accuracy of the proposed recursions and illustrate posterior inference and sampling for representative SDE-induced kernels.