arXiv subjects

El Mahdi Khribch

Publications and source records attributed to El Mahdi Khribch.

3 recordsLinked to original sources

arXiv2026

Convergence of Statistical Estimators via Mutual Information Bounds

Recent advances in statistical learning theory have revealed profound connections between mutual information (MI) bounds, PAC-Bayesian theory, and Bayesian nonparametrics. This work introduces a mutual information bound for statistical models, and derives from it convergence rates for fractional posteriors, for their variational approximations, and for the maximum likelihood estimator. The observations are assumed independent but not identically distributed, and the model is not assumed well-specified, so that the bounds are oracle inequalities and cover regression with a fixed or a conditioned design; the independent and identically distributed, well-specified case is recovered by dropping an index. We illustrate the method on two applications. In the Gaussian sequence model the rate is minimax in both the radius of the Sobolev ball and the sample size, which only appears through its product with the temperature. In logistic regression, where the model is not conjugate and the variational approximation is computed by a stochastic gradient method, the bound applies to the output of the algorithm rather than to an idealized minimizer, and attains the parametric order in the Renyi risk with no logarithmic factor, the statistical and the optimization error being separated.

stat.ML↗

arXiv2026

Convex Order Comparisons for Sub-Gamma Random Variables

Recent work has shown that sub-Gaussian random variables are dominated in convex order by a sharp multiple of a Gaussian. We study the analogous question for the sub-Gamma class underlying Bernstein's inequality, with the Laplace law as the majorant. Here the moment generating function is controlled by a Bernstein-type bound over a finite range of frequencies, rather than by a purely quadratic bound. We derive a variational formula for the optimal multiple, show that it is strictly larger than the natural scale $σ\vee α$, and prove that this value is sharp, being attained by an asymmetric two-point distribution. We then turn to the sub-exponential class, which has a quadratic bound as in the sub-Gaussian case, but only over a bounded range as in the sub-Gamma case. Interestingly, the sub-exponential class displays a different behavior: the optimal multiple is exactly $σ\vee α$, but is attained only when $α\leq σ$; when $α> σ$, the constant remains sharp, but equality cannot hold for any non-affine convex function. Both results rely on a sharp finite-range variant of the Kearns-Saul inequality, which is of independent interest.

math.PR↗

arXiv2026

On importance sampling and independent Metropolis-Hastings with an unbounded weight function

Importance sampling and independent Metropolis-Hastings are among the fundamental building blocks of Monte Carlo methods. Both require a proposal distribution that globally approximates the target distribution, and pointwise evaluation of the Radon-Nikodym derivative of the target distribution relative to the proposal, also called the weight function. We study the bias of importance sampling and independent Metropolis-Hastings, without assuming that the weight function is bounded. We show that the common random numbers coupling of independent Metropolis-Hastings is maximal. Using that coupling, we derive polynomial bounds on the total variation distance of the chain to its target distribution. We further consider bias removal techniques using couplings, and provide conditions under which the resulting unbiased estimators have finite moments, and under which their efficiency is comparable to that of importance sampling. Experiments illustrate unbiased estimators of the inverse of a normalizing constant, estimators of nested expectations, and combination of importance sampling with robust mean estimation methods.

math.ST↗