A Better Good-Turing Estimator for Sequence Probabilities

Aaron B. Wagner; Pramod Viswanath; Sanjeev R. Kulkarni

Home/Archive/arXiv

arXiv · 0704.1455

A Better Good-Turing Estimator for Sequence Probabilities

Aaron B. Wagner·Pramod Viswanath·Sanjeev R. Kulkarni

View original source ↗Read source document ↗Export metadata ↓

Abstract

We consider the problem of estimating the probability of an observed string drawn i.i.d. from an unknown distribution. The key feature of our study is that the length of the observed string is assumed to be of the same order as the size of the underlying alphabet. In this setting, many letters are unseen and the empirical distribution tends to overestimate the probability of the observed letters. To overcome this problem, the traditional approach to probability estimation is to use the classical Good-Turing estimator. We introduce a natural scaling model and use it to show that the Good-Turing sequence probability estimator is not consistent. We then introduce a novel sequence probability estimator that is indeed consistent under the natural scaling model.

Explore related subjects

cs.IT math.IT

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Aaron B. Wagner, Pramod Viswanath, Sanjeev R. Kulkarni. 2007-04-21. A Better Good-Turing Estimator for Sequence Probabilities. https://arxiv.org/abs/0704.1455

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Phased Ultra Massive Array (PUMA)

This paper proposes a novel multiple-access framework, termed the phased ultra massive antenna array (PUMA), which exploits the distinctive spatial flexibility of fluid antenna systems (FAS) at the user equipment (UE). Building upon fluid antenna multiple access (FAMA) and compact ultra-massive antenna array (CUMA), PUMA incorporates a phased array for signal aggregation. This architecture enables the UE to inherently mitigate co-user interference within the spatial domain without necessitating channel state information (CSI) for precoding at the base station (BS) or complex interference cancellation at each UE. A primary advantage of PUMA lies in its hardware efficiency: by implementing phase shifting and signal combining in the analog domain, it achieves high antenna gain while requiring only a minimal number of radio-frequency (RF) chains, potentially a single RF chain. Comprehensive theoretical analysis of the achievable data rate is provided, complemented by extensive simulations that validate the framework. The results demonstrate that PUMA markedly outperforms FAMA and CUMA architectures, particularly for UEs with a single RF chain, offering a robust and scalable solution for interference-insensitive massive connectivity in sixth-generation (6G) systems.

cs.IT↗

arXiv2026

Identification for Inverse Gaussian Channels

We derive lower and upper bounds on the identification capacity of inverse Gaussian (IG) channels, a fundamental model for molecular communications in fluid environments. The analysis considers deterministic encoding schemes under a peak time constraint and characterizes the asymptotic growth of optimal codebook sizes. In the converse, by leveraging the Borel-Cantelli lemma and the super-exponential near-zero tail behavior of the IG distribution, we establish an almost-sure lower bound on the first-passage time of drifting diffusing molecules. This leads to the conclusion that the identification capacity exhibits super-exponential growth with the codeword length $n$, i.e., $\sim 2^{(n \log n)R_n}$, where $R_n$ is the coding rate.

cs.IT↗

arXiv2026

Polar Complexity: A New Descriptive Complexity with Applications to Source and Joint Source-Channel Coding

This paper first presents a new approach to evaluating the descriptive complexity of finite-length binary sequences. Specifically, we investigate the sequence-wise recovery behavior induced by polar compression and successive cancellation decoding (SCD), and define the polar complexity of a sequence as the minimum polar-compression length (PCL) required for its exact reconstruction. To compute the polar complexity efficiently, we further develop both a bisection-search algorithm and a low-complexity estimation method. We then propose a polar-based two-stage source coding scheme, in which each source sequence is represented by its polar complexity followed by the corresponding polar-compressed sequence. The proposed scheme is strictly lossless and prefix-free. In addition, for BMSs, the normalized average compression length of the proposed scheme can asymptotically approach the source entropy under certain conditions. Simulation results further demonstrate that the scheme can operate without prior knowledge of the source statistics and remains robust across different source distributions. Finally, we integrate the proposed polar source coding with polar channel coding to develop an adaptive double-polar joint source-channel coding (JSCC) scheme, where the encoder and decoder share a predefined set of candidate PCLs to balance error performance and decoding complexity. We formulate the design of the candidate-PCL set as an optimization problem and solve it efficiently via dynamic programming. Simulation results show that the proposed adaptive double-polar JSCC scheme provides a flexible performance-complexity tradeoff and outperforms existing polar-code-based JSCC baselines.

cs.IT↗