Exact Transport-Information Correspondence for Distribution-Adaptive Discretization

Abstract

Discretization of continuous probability distributions is an operation on the sample space, and the structure induced by this operation cannot generally be determined solely from information geometry. In our recent work on unavoidable canonical nonlinearity (UCN), we established a non-arbitrary connection between the transport cost associated with discretization and the information-geometric indistinguishability of continuous probability distributions. Through the transport-information correspondence, the UCN quantify the information-theoretic cost due to discretization as the expected Kullback-Leibler divergence between nearby continuous distributions associated with the discretization geometry. In the original UCN construction, discretization is implemented by transporting the probability mass within each cell to its geometric centroid. The resulting transport-information correspondence, however, is derived under specific conditions involving the cell geometry, spatial coverage, and the vanishing-cell-scale limit. In this work, we introduce an alternative local-map discretization scheme in which the probability mass within each cell is transported to its own probability-weighted centroid. This change in the representative-point selection enables an exact expression for the transport cost of each cell in terms of the covariance matrix of its conditional probability distribution and the Fisher metric of the underlying continuous family. We establish an exact correspondence between the resulting transport cost and the expectation of the associated Fisher-induced quadratic form, without requiring uniform cell geometry, complete coverage of the sample space, or an infinitesimal cell-scale limit. The proposed construction thus generalizes the transport-information correspondence to finite-scale discretization using arbitrary collections of non-overlapping bounded convex sets.