arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.02558

Bias-Corrected Estimators for Joint Entropy, Conditional Entropy, and Mutual Information in the Discrete Bivariate Case: Theory and an Application to Motor Insurance

Amadou Diadie Ba
View original source ↗Read source document ↗Export metadata ↓

Abstract

This paper studies the finite-sample bias of plug-in estimators for joint entropy, conditional entropy, and mutual information for finitely supported discrete random variables. Using the univariate reduction framework developed in our companion works, we derive explicit first-order bias formulas. For a pair (X,Y) with support sizes r and s, the biases are respectively -(rs-1)/(2n), -r(s-1)/(2n), and (r-1)(s-1)/(2n). We then propose bias-corrected estimators for the three measures and prove that the uncorrected and corrected versions are asymptotically equivalent, differing by a deterministic term of order O(1/n); consequently, they share the same asymptotic distribution. A simulation study validates the theoretical results, including a dedicated independence study and the relationship with Wilks' theorem. An application to a motor insurance portfolio, where X is the driver's age class and Y the claim severity class, illustrates the improvement brought by the bias correction for risk classification.

Explore related subjects

math.STstat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Amadou Diadie Ba. 2026-10-01. Bias-Corrected Estimators for Joint Entropy, Conditional Entropy, and Mutual Information in the Discrete Bivariate Case: Theory and an Application to Motor Insurance. https://arxiv.org/abs/2610.02558

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

KL Convergence Guarantees for Score diffusion models under minimal data assumptions

Diffusion models are a new class of generative models that revolve around the estimation of the score function associated with a stochastic differential equation. Subsequent to its acquisition, the approximated score function is then harnessed to simulate the corresponding time-reversal process, ultimately enabling the generation of approximate data samples. Despite their evident practical significance these models carry, a notable challenge persists in the form of a lack of comprehensive quantitative results, especially in scenarios involving non-regular scores and estimators. In almost all reported bounds in Kullback Leibler (KL) divergence, it is assumed that either the score function or its approximation is Lipschitz uniformly in time. However, this condition is very restrictive in practice or appears to be difficult to establish. To circumvent this issue, previous works mainly focused on establishing convergence bounds in KL for an early stopped version of the diffusion model and a smoothed version of the data distribution, or assuming that the data distribution is supported on a compact manifold. These explorations have led to interesting bounds in either Wasserstein or Fortet-Mourier metrics. However, the question remains about the relevance of such early-stopping procedure or compactness conditions. In particular, if there exist a natural and mild condition ensuring explicit and sharp convergence bounds in KL. In this article, we tackle the aforementioned limitations by focusing on score diffusion models with fixed step size stemming from the Ornstein-Uhlenbeck semigroup and its kinetic counterpart. Our study provides a rigorous analysis, yielding simple, improved and sharp convergence bounds in KL applicable to any data distribution with finite Fisher information with respect to the standard Gaussian distribution.

math.ST↗
arXiv2026

Estimation of conditional inequality curves and measures via estimating the conditional quantile function

In the paper conditional inequality curves and measures are proposed which allow us to describe the inequality/concentration of the conditional distribution of the feature we are interested in with respect to certain continuous variables. Moreover, for a graphical illustration of the change in values of the proposed conditional indices, a curve of conditional inequality measures is introduced. To estimate the curves and measures, a new method is proposed to estimate the conditional quantile function. This method uses quantile regression estimates for a given set of quantile orders, followed by isotonic regression on the estimated regression coefficients to ensure that the estimated conditional quantile function is nondecreasing. The consistency of the proposed estimators is proved while their finite sample performance is evaluated through simulation studies and compared with existing approaches. Finally, practical application of conditional curves and measures is demonstrated by determining estimated curves of conditional salary inequalities with respect to years of experience in different employee tenure groups, based on some real data. The code used to prepare the simulation results presented in this paper is available in a dedicated GitHub repository.

math.ST↗
arXiv2026

Tracy-Widom, Gaussian, and Bootstrap: Approximations for Leading Eigenvalues in High-Dimensional PCA

Under certain conditions, the largest eigenvalue of a sample covariance matrix undergoes a well-known phase transition when the sample size $n$ and data dimension $p$ diverge proportionally. In the subcritical regime, this eigenvalue has fluctuations of order $n^{-2/3}$ that can be approximated by a Tracy-Widom distribution, while in the supercritical regime, it has fluctuations of order $n^{-1/2}$ that can be approximated with a Gaussian distribution. However, the statistical problem of determining which regime underlies a given dataset is far from resolved. We develop a new testing framework and procedure to address this problem. In particular, we demonstrate that the procedure has an asymptotically controlled level, and that it is power consistent for certain alternatives. Also, this testing procedure enables the design a new bootstrap method for approximating the distributions of functionals of the leading sample eigenvalues within the subcritical regime -- which is the first such method that is supported by theoretical guarantees.

math.ST↗