Optimal tradeoffs between variance and sensitivity, via Hellinger-Kantorovich geometry

Abstract

While the goal of classical statistical theory is to minimize the variance of unbiased estimators, an additional goal of some recent research is to minimize or constrain other notions of the instability of estimators. In particular, a subset of the authors recently introduced a notion of instability termed "sensitivity", which represents the expected amount that an estimator would change if its inputs were infinitesimally perturbed. In this work, we study the multiobjective problem of minimizing both the variance and sensitivity over unbiased estimators in a given statistical model, and we aim to determine Pareto optimal tradeoffs between these two objectives. Our main results show that the Pareto front for these tradeoffs is precisely determined by endowing the statistical model with the Hellinger-Kantorovich geometry, which interpolates between the Hellinger and Wasserstein geometries that are known to be associated, respectively, with the separate minimization of variance and sensitivity. We also develop a $Z$-estimation technique which constructs estimators that asymptotically achieve Pareto optimal pairs of variance and sensitivity. While computing the Pareto front and constructing the resulting estimators in practice requires solving a continuum of partial differential equations, we fully execute this program in some parametric examples of interest where the analysis is already non-trivial. Our results also shed light on some classical statistical problems, for example by showing that the maximum likelihood estimator achieves the smallest asymptotic sensitivity among all estimators achieving the optimal asymptotic variance.