arXiv subjects

Baris Nakiboglu

Publications and source records attributed to Baris Nakiboglu.

13 recordsLinked to original sources

arXiv2021

On the Existence of the Augustin Mean

The existence of a unique Augustin mean and its invariance under the Augustin operator are established for arbitrary input distributions with finite Augustin information for channels with countably generated output $\sigma$-algebras. The existence is established by representing the conditional R\'enyi divergence as a lower semicontinuous and convex functional in an appropriately chosen uniformly convex space and then invoking the Banach--Saks property in conjunction with the lower semicontinuity and the convexity. A new family of operators is proposed to establish the invariance of the Augustin mean under the Augustin operator for orders greater than one. Some members of this new family strictly decrease the conditional R\'enyi divergence, when applied to the second argument of the divergence, unless the second argument is a fixed point of the Augustin operator.

cs.IT↗

arXiv2020

Refined Strong Converse for the Constant Composition Codes

A strong converse bound for constant composition codes of the form $P_{e}^{(n)} \geq 1- A n^{-0.5(1-E_{sc}'(R,W,p))} e^{-n E_{sc}(R,W,p)}$ is established using the Berry-Esseen theorem through the concepts of Augustin information and Augustin mean, where $A$ is a constant determined by the channel $W$, the composition $p$, and the rate $R$, i.e., $A$ does not depend on the block length $n$.

cs.IT↗

arXiv2019

A Simple Derivation of the Refined Sphere Packing Bound Under Certain Symmetry Hypotheses

A judicious application of the Berry-Esseen theorem via suitable Augustin information measures is demonstrated to be sufficient for deriving the sphere packing bound with a prefactor that is $\mathit{\Omega}\left(n^{-0.5(1-E_{sp}'(R))}\right)$ for all codes on certain families of channels -- including the Gaussian channels and the non-stationary Renyi symmetric channels -- and for the constant composition codes on stationary memoryless channels. The resulting non-asymptotic bounds have definite approximation error terms. As a preliminary result that might be of interest on its own, the trade-off between type I and type II error probabilities in the hypothesis testing problem with (possibly non-stationary) independent samples is determined up to some multiplicative constants, assuming that the probabilities of both types of error are decaying exponentially with the number of samples, using the Berry-Esseen theorem.

cs.IT↗

arXiv2018

The Sphere Packing Bound for DSPCs with Feedback a la Augustin

Establishing the sphere packing bound for block codes on the discrete stationary product channels with feedback ---which are commonly called the discrete memoryless channels with feedback--- was considered to be an open problem until recently, notwithstanding the proof sketch provided by Augustin in 1978. A complete proof following Augustin's proof sketch is presented, to demonstrate its adequacy and to draw attention to two novel ideas it employs. These novel ideas (i.e., the Augustin's averaging and the use of subblocks) are likely to be applicable in other communication problems for establishing impossibility results.

cs.IT↗

arXiv2018

The Sphere Packing Bound For Memoryless Channels

Sphere packing bounds (SPBs) ---with prefactors that are polynomial in the block length--- are derived for codes on two families of memoryless channels using Augustin's method: (possibly non-stationary) memoryless channels with (possibly multiple) additive cost constraints and stationary memoryless channels with convex constraints on the composition (i.e. empirical distribution, type) of the input codewords. A variant of Gallager's bound is derived in order to show that these sphere packing bounds are tight in terms of the exponential decay rate of the error probability with the block length under mild hypotheses.

cs.IT↗

arXiv2018

The Augustin Capacity and Center

For any channel, the existence of a unique Augustin mean is established for any positive order and probability mass function on the input set. The Augustin mean is shown to be the unique fixed point of an operator defined in terms of the order and the input distribution. The Augustin information is shown to be continuously differentiable in the order. For any channel and convex constraint set with finite Augustin capacity, the existence of a unique Augustin center and the associated van Erven-Harremoes bound are established. The Augustin-Legendre (A-L) information, capacity, center, and radius are introduced and the latter three are proved to be equal to the corresponding Renyi-Gallager quantities. The equality of the A-L capacity to the A-L radius for arbitrary channels and the existence of a unique A-L center for channels with finite A-L capacity are established. For all interior points of the feasible set of cost constraints, the cost constrained Augustin capacity and center are expressed in terms of the A-L capacity and center. Certain shift invariant families of probabilities and certain Gaussian channels are analyzed as examples.

cs.IT↗

arXiv2016

The Sphere Packing Bound via Augustin's Method

A sphere packing bound (SPB) with a prefactor that is polynomial in the block length $n$ is established for codes on a length $n$ product channel $W_{[1,n]}$ assuming that the maximum order $1/2$ Renyi capacity among the component channels, i.e. $\max_{t\in[1,n]} C_{1/2,W_{t}}$, is $\mathit{O}(\ln n)$. The reliability function of the discrete stationary product channels with feedback is bounded from above by the sphere packing exponent. Both results are proved by first establishing a non-asymptotic SPB. The latter result continues to hold under a milder stationarity hypothesis.

cs.IT↗

arXiv2016

The Renyi Capacity and Center

Renyi's information measures ---the Renyi information, mean, capacity, radius, and center--- are analyzed relying on the elementary properties of the Renyi divergence and the power means. The van Erven-Harremoes conjecture is proved for any positive order and for any set of probability measures on a given measurable space and a generalization of it is established for the constrained variant of the problem. The finiteness of the order $\alpha$ Renyi capacity is shown to imply the continuity of the Renyi capacity on $(0,\alpha]$ and the uniform equicontinuity of the Renyi information, both as a family of functions of the order indexed by the priors and as a family of functions of the prior indexed by the orders. The Renyi capacities and centers of various families of Poisson processes are derived as examples.

cs.IT↗

arXiv2011

Bit-wise Unequal Error Protection for Variable Length Block Codes with Feedback

The bit-wise unequal error protection problem, for the case when the number of groups of bits $\ell$ is fixed, is considered for variable length block codes with feedback. An encoding scheme based on fixed length block codes with erasures is used to establish inner bounds to the achievable performance for finite expected decoding time. A new technique for bounding the performance of variable length block codes is used to establish outer bounds to the performance for a given expected decoding time. The inner and the outer bounds match one another asymptotically and characterize the achievable region of rate-exponent vectors, completely. The single message message-wise unequal error protection problem for variable length block codes with feedback is also solved as a necessary step on the way.

cs.IT↗

arXiv2009

Error-and-Erasure Decoding for Block Codes with Feedback

Inner and outer bounds are derived on the optimal performance of fixed length block codes on discrete memoryless channels with feedback and errors-and-erasures decoding. First an inner bound is derived using a two phase encoding scheme with communication and control phases together with the optimal decoding rule for the given encoding scheme, among decoding rules that can be represented in terms of pairwise comparisons between the messages. Then an outer bound is derived using a generalization of the straight-line bound to errors-and-erasures decoders and the optimal error exponent trade off of a feedback encoder with two messages. In addition upper and lower bounds are derived, for the optimal erasure exponent of error free block codes in terms of the rate. Finally we present a proof of the fact that the optimal trade off between error exponents of a two message code does not increase with feedback on DMCs.

cs.IT↗

arXiv2008

Variations on a theme by Schalkwijk and Kailath

Schalkwijk and Kailath (1966) developed a class of block codes for Gaussian channels with ideal feedback for which the probability of decoding error decreases as a second-order exponent in block length for rates below capacity. This well-known but surprising result is explained and simply derived here in terms of a result by Elias (1956) concerning the minimum mean-square distortion achievable in transmitting a single Gaussian random variable over multiple uses of the same Gaussian channel. A simple modification of the Schalkwijk-Kailath scheme is then shown to have an error probability that decreases with an exponential order which is linearly increasing with block length. In the infinite bandwidth limit, this scheme produces zero error probability using bounded expected energy at all rates below capacity. A lower bound on error probability for the finite bandwidth case is then derived in which the error probability decreases with an exponential order which is linearly increasing in block length at the same rate as the upper bound.

cs.IT↗

arXiv2008

Unequal Error Protection: An Information Theoretic Perspective

An information theoretic framework for unequal error protection is developed in terms of the exponential error bounds. The fundamental difference between the bit-wise and message-wise unequal error protection (UEP) is demonstrated, for fixed length block codes on DMCs without feedback. Effect of feedback is investigated via variable length block codes. It is shown that, feedback results in a significant improvement in both bit-wise and message-wise UEP (except the single message case for missed detection). The distinction between false-alarm and missed-detection formalizations for message-wise UEP is also considered. All results presented are at rates close to capacity.

cs.IT↗

arXiv2006

A Simple Converse of Burnashev's Reliability

In a remarkable paper published in 1976, Burnashev determined the reliability function of variable-length block codes over discrete memoryless channels with feedback. Subsequently, an alternative achievability proof was obtained by Yamamoto and Itoh via a particularly simple and instructive scheme. Their idea is to alternate between a communication and a confirmation phase until the receiver detects the codeword used by the sender to acknowledge that the message is correct. We provide a converse that parallels the Yamamoto-Itoh achievability construction. Besides being simpler than the original, the proposed converse suggests that a communication and a confirmation phase are implicit in any scheme for which the probability of error decreases with the largest possible exponent. The proposed converse also makes it intuitively clear why the terms that appear in Burnashev's exponent are necessary.

cs.IT↗