arXiv Science⌕ Search

arXiv subjects

Terje Mildner

Publications and source records attributed to Terje Mildner.

3 recordsLinked to original sources

Asymptotics of Gibbs Posterior Measures on Infinite-Dimensional Hypothesis Spaces

We prove almost sure contraction rates and consistency for Gibbs posterior measures on arbitrary hypothesis spaces, in general infinite dimensional, under dependent data, nonadditive and time-dependent losses and learning rates, misspecified models, and unattainable minimisers of the limiting objective. In particular, we develop a novel proof approach in contraction rate theory by studying the asymptotics of Gibbs measures through Lorentz spaces, real interpolation theory, and Laplace transforms dispensing with testing or variational formulations which are unavailable in the considered generality. As a consequence we obtain almost sure contraction rates for dominated Bayes and fractional Bayes posterior measures in the same generality, extending existing theory on misspecified Bayesian inference. Furthermore, we demonstrate our theoretical results on novel applications to Gibbs posteriors for Dirichlet process priors with a nonadditive Wasserstein metric loss function under ergodic, non-Markovian observables.

math.ST↗

Rates of Convergence of Generalised Variational Inference Posteriors under Prior Misspecification

We prove rates of convergence and robustness to prior misspecification within a Generalised Variational Inference (GVI) framework with bounded divergences. This addresses a significant open challenge for GVI and Federated GVI that employ a different divergence to the Kullback-Leibler under prior misspecification, operate within a subset of possible probability measures, and result in intractable posteriors. Our theoretical contributions extend to misspecified priors that lead to inconsistent Bayes posteriors. In particular, we are able to establish sufficient conditions for existence and uniqueness of GVI posteriors on arbitrary Polish spaces, prove that the GVI posterior measure concentrates on a neighbourhood of loss minimisers, and extend this to rates of convergence regardless of the prior measure.

math.ST↗

Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework

We introduce FedGVI, a probabilistic Federated Learning (FL) framework that is robust to both prior and likelihood misspecification. FedGVI addresses limitations in both frequentist and Bayesian FL by providing unbiased predictions under model misspecification, with calibrated uncertainty quantification. Our approach generalises previous FL approaches, specifically Partitioned Variational Inference (Ashman et al., 2022), by allowing robust and conjugate updates, decreasing computational complexity at the clients. We offer theoretical analysis in terms of fixed-point convergence, optimality of the cavity distribution, and provable robustness to likelihood misspecification. Further, we empirically demonstrate the effectiveness of FedGVI in terms of improved robustness and predictive performance on multiple synthetic and real world classification data sets.

cs.LG↗