Geometry and universal scaling of Pareto-optimal signal compression

Abstract

I investigate the generic problem of lossy compression of a fluctuating stochastic signal $X$ into a discrete representation $Z$ through optimal thresholding. The signal modulates transition rates of a two-state system described by a binary variable $Y$. Optimising the retained mutual information between $Z$ and $Y$ under a constraint on fixed encoding cost of $Z$ reveals Pareto-optimal trade-offs, determined numerically using genetic algorithms. In the small-noise regime, these fronts are either concave or exhibit piecewise convex ``intrusions'' separated by first-order transitions in the optimal protocol. An analytical high-rate expansion shows that the optimal threshold density follows a universal cube-root scaling with the product of the prior distribution and the Fisher information associated with the response, which holds qualitatively even for few discrete states. Extending the analysis to non-Gaussian fluctuations reveals that for some parameters optimal encoders can yield strictly better information-cost trade-offs than Gaussian surrogates, meaning the same information content can often be achieved with fewer discrete readout states.