arXiv subjects

Henry D. Pfister

Publications and source records attributed to Henry D. Pfister.

108 records · Page 6Linked to original sources

arXiv2011

Convergence of Weighted Min-Sum Decoding Via Dynamic Programming on Trees

Applying the max-product (and belief-propagation) algorithms to loopy graphs is now quite popular for best assignment problems. This is largely due to their low computational complexity and impressive performance in practice. Still, there is no general understanding of the conditions required for convergence and/or the optimality of converged solutions. This paper presents an analysis of both attenuated max-product (AMP) decoding and weighted min-sum (WMS) decoding for LDPC codes which guarantees convergence to a fixed point when a weight parameter, β, is sufficiently small. It also shows that, if the fixed point satisfies some consistency conditions, then it must be both the linear-programming (LP) and maximum-likelihood (ML) solution. For (dv,dc)-regular LDPC codes, the weight must satisfy β(dv-1) \leq 1 whereas the results proposed by Frey and Koetter require instead that β(dv-1)(dc-1) < 1. A counterexample which shows a fixed point might not be the ML solution if β(dv-1) > 1 is also given. Finally, connections are explored with recent work by Arora et al. on the threshold of LP decoding.

cs.IT↗

arXiv2011

Analysis of Verification-based Decoding on the q-ary Symmetric Channel for Large q

We discuss and analyze a list-message-passing decoder with verification for low-density parity-check (LDPC) codes on the q-ary symmetric channel (q-SC). Rather than passing messages consisting of symbol probabilities, this decoder passes lists of possible symbols and marks some lists as verified. The density evolution (DE) equations for this decoder are derived and used to compute decoding thresholds. If the maximum list size is unbounded, then we find that any capacity-achieving LDPC code for the binary erasure channel can be used to achieve capacity on the q-SC for large q. The decoding thresholds are also computed via DE for the case where each list is truncated to satisfy a maximum list size constraint. Simulation results are also presented to confirm the DE results. During the simulations, we observed differences between two verification-based decoding algorithms, introduced by Luby and Mitzenmacher, that were implicitly assumed to be identical. In this paper, we provide an analysis of the node-based algorithms from that paper and verify that it matches simulation results. The probability of false verification (FV) is also considered and techniques are discussed to mitigate the FV. Optimization of the degree distribution is also used to improve the threshold for a fixed maximum list size. Finally, the proposed algorithm is compared with a variety of other algorithms using both density evolution thresholds and simulation results.

cs.IT↗

arXiv2011

Universality for the Noisy Slepian-Wolf Problem Via Spatial Coupling

We consider a noisy Slepian-Wolf problem where two correlated sources are separately encoded and transmitted over two independent binary memoryless symmetric channels. Each channel capacity is assumed to be characterized by a single parameter which is not known at the transmitter. The receiver has knowledge of both the source correlation and the channel parameters. We call a system universal if it retains near-capacity performance without channel knowledge at the transmitter. Kudekar et al. recently showed that terminated low-density parity-check (LDPC) convolutional codes (a.k.a. spatially-coupled LDPC ensembles) can have belief-propagation thresholds that approach their maximum a-posteriori thresholds. This was proven for binary erasure channels and shown empirically for binary memoryless symmetric channels. They also conjectured that the principle of spatial coupling is very general and the phenomenon of threshold saturation applies to a very broad class of graphical models. In this work, we derive an area theorem for the joint decoder and empirically show that threshold saturation occurs for this problem. As a result, we demonstrate near-universal performance for this problem using the proposed spatially-coupled coding system. A similar result is also discussed briefly for the 2-user multiple-access channel.

cs.IT↗

arXiv2011

An Iterative Joint Linear-Programming Decoding of LDPC Codes and Finite-State Channels

In this paper, we introduce an efficient iterative solver for the joint linear-programming (LP) decoding of low-density parity-check (LDPC) codes and finite-state channels (FSCs). In particular, we extend the approach of iterative approximate LP decoding, proposed by Vontobel and Koetter and explored by Burshtein, to this problem. By taking advantage of the dual-domain structure of the joint decoding LP, we obtain a convergent iterative algorithm for joint LP decoding whose structure is similar to BCJR-based turbo equalization (TE). The result is a joint iterative decoder whose complexity is similar to TE but whose performance is similar to joint LP decoding. The main advantage of this decoder is that it appears to provide the predictability of joint LP decoding and superior performance with the computational complexity of TE.

cs.IT↗

arXiv2010

On Multiple Decoding Attempts for Reed-Solomon Codes: A Rate-Distortion Approach

One popular approach to soft-decision decoding of Reed-Solomon (RS) codes is based on using multiple trials of a simple RS decoding algorithm in combination with erasing or flipping a set of symbols or bits in each trial. This paper presents a framework based on rate-distortion (RD) theory to analyze these multiple-decoding algorithms. By defining an appropriate distortion measure between an error pattern and an erasure pattern, the successful decoding condition, for a single errors-and-erasures decoding trial, becomes equivalent to distortion being less than a fixed threshold. Finding the best set of erasure patterns also turns into a covering problem which can be solved asymptotically by rate-distortion theory. Thus, the proposed approach can be used to understand the asymptotic performance-versus-complexity trade-off of multiple errors-and-erasures decoding of RS codes. This initial result is also extended a few directions. The rate-distortion exponent (RDE) is computed to give more precise results for moderate blocklengths. Multiple trials of algebraic soft-decision (ASD) decoding are analyzed using this framework. Analytical and numerical computations of the RD and RDE functions are also presented. Finally, simulation results show that sets of erasure patterns designed using the proposed methods outperform other algorithms with the same number of decoding trials.

cs.IT↗

arXiv2010

The Effect of Spatial Coupling on Compressive Sensing

Recently, it was observed that spatially-coupled LDPC code ensembles approach the Shannon capacity for a class of binary-input memoryless symmetric (BMS) channels. The fundamental reason for this was attributed to a "threshold saturation" phenomena derived by Kudekar, Richardson and Urbanke. In particular, it was shown that the belief propagation (BP) threshold of the spatially coupled codes is equal to the maximum a posteriori (MAP) decoding threshold of the underlying constituent codes. In this sense, the BP threshold is saturated to its maximum value. Moreover, it has been empirically observed that the same phenomena also occurs when transmitting over more general classes of BMS channels. In this paper, we show that the effect of spatial coupling is not restricted to the realm of channel coding. The effect of coupling also manifests itself in compressed sensing. Specifically, we show that spatially-coupled measurement matrices have an improved sparsity to sampling threshold for reconstruction algorithms based on verification decoding. For BP-based reconstruction algorithms, this phenomenon is also tested empirically via simulation. At the block lengths accessible via simulation, the effect is quite small and it seems that spatial coupling is not providing the gains one might expect. Based on the threshold analysis, however, we believe this warrants further study.

cs.IT↗

arXiv2010

LDPC Code Design for Transmission of Correlated Sources Across Noisy Channels Without CSIT

We consider the problem of transmitting correlated data after independent encoding to a central receiver through orthogonal channels. We assume that the channel state information is not known at the transmitter. The receiver has access to both the source correlation and the channel state information. We provide a generic framework for analyzing the performance of joint iterative decoding, using density evolution. Using differential evolution, we design punctured systematic LDPC codes to maximize the region of achievable channel conditions, with joint iterative decoding. The main contribution of this paper is to demonstrate that properly designed LDPC can perform well simultaneously over a wide range of channel parameters.

cs.IT↗

arXiv2010

IMP: A Message-Passing Algorithmfor Matrix Completion

A new message-passing (MP) method is considered for the matrix completion problem associated with recommender systems. We attack the problem using a (generative) factor graph model that is related to a probabilistic low-rank matrix factorization. Based on the model, we propose a new algorithm, termed IMP, for the recovery of a data matrix from incomplete observations. The algorithm is based on a clustering followed by inference via MP (IMP). The algorithm is compared with a number of other matrix completion algorithms on real collaborative filtering (e.g., Netflix) data matrices. Our results show that, while many methods perform similarly with a large number of revealed entries, the IMP algorithm outperforms all others when the fraction of observed entries is small. This is helpful because it reduces the well-known cold-start problem associated with collaborative filtering (CF) systems in practice.

cs.IT↗

arXiv2010

On the Queueing Behavior of Random Codes over a Gilbert-Elliot Erasure Channel

This paper considers the queueing performance of a system that transmits coded data over a time-varying erasure channel. In our model, the queue length and channel state together form a Markov chain that depends on the system parameters. This gives a framework that allows a rigorous analysis of the queue as a function of the code rate. Most prior work in this area either ignores block-length (e.g., fluid models) or assumes error-free communication using finite codes. This work enables one to determine when such assumptions provide good, or bad, approximations of true behavior. Moreover, it offers a new approach to optimize parameters and evaluate performance. This can be valuable for delay-sensitive systems that employ short block lengths.

cs.IT↗

arXiv2010

On the Joint Decoding of LDPC Codes and Finite-State Channels via Linear Programming

In this paper, the linear programming (LP) decoder for binary linear codes, introduced by Feldman, et al. is extended to joint-decoding of binary-input finite-state channels. In particular, we provide a rigorous definition of LP joint-decoding pseudo-codewords (JD-PCWs) that enables evaluation of the pairwise error probability between codewords and JD-PCWs. This leads naturally to a provable upper bound on decoder failure probability. If the channel is a finite-state intersymbol interference channel, then the LP joint decoder also has the maximum-likelihood (ML) certificate property and all integer valued solutions are codewords. In this case, the performance loss relative to ML decoding can be explained completely by fractional valued JD-PCWs.

cs.IT↗

arXiv2010

A Rate-Distortion Exponent Approach to Multiple Decoding Attempts for Reed-Solomon Codes

Algorithms based on multiple decoding attempts of Reed-Solomon (RS) codes have recently attracted new attention. Choosing decoding candidates based on rate-distortion (R-D) theory, as proposed previously by the authors, currently provides the best performance-versus-complexity trade-off. In this paper, an analysis based on the rate-distortion exponent (RDE) is used to directly minimize the exponential decay rate of the error probability. This enables rigorous bounds on the error probability for finite-length RS codes and leads to modest performance gains. As a byproduct, a numerical method is derived that computes the rate-distortion exponent for independent non-identical sources. Analytical results are given for errors/erasures decoding.

cs.IT↗

arXiv2010

Message-Passing Inference on a Factor Graph for Collaborative Filtering

This paper introduces a novel message-passing (MP) framework for the collaborative filtering (CF) problem associated with recommender systems. We model the movie-rating prediction problem popularized by the Netflix Prize, using a probabilistic factor graph model and study the model by deriving generalization error bounds in terms of the training error. Based on the model, we develop a new MP algorithm, termed IMP, for learning the model. To show superiority of the IMP algorithm, we compare it with the closely related expectation-maximization (EM) based algorithm and a number of other matrix completion algorithms. Our simulation results on Netflix data show that, while the methods perform similarly with large amounts of data, the IMP algorithm is superior for small amounts of data. This improves the cold-start problem of the CF systems in practice. Another advantage of the IMP algorithm is that it can be analyzed using the technique of density evolution (DE) that was originally developed for MP decoding of error-correcting codes.

cs.IT↗

arXiv2010

The Capacity of Finite-State Channels in the High-Noise Regime

This paper considers the derivative of the entropy rate of a hidden Markov process with respect to the observation probabilities. The main result is a compact formula for the derivative that can be evaluated easily using Monte Carlo methods. It is applied to the problem of computing the capacity of a finite-state channel (FSC) and, in the high-noise regime, the formula has a simple closed-form expression that enables series expansion of the capacity of a FSC. This expansion is evaluated for a binary-symmetric channel under a (0,1) run-length limited constraint and an intersymbol-interference channel with Gaussian noise.

cs.IT↗

arXiv2009

Modulation Codes for Flash Memory Based on Load-Balancing Theory

In this paper, we consider modulation codes for practical multilevel flash memory storage systems with cell levels. Instead of maximizing the lifetime of the device [Ajiang-isit07-01, Ajiang-isit07-02, Yaakobi_verdy_siegel_wolf_allerton08, Finucane_Liu_Mitzenmacher_aller08], we maximize the average amount of information stored per cell-level, which is defined as storage efficiency. Using this framework, we show that the worst-case criterion [Ajiang-isit07-01, Ajiang-isit07-02, Yaakobi_verdy_siegel_wolf_allerton08] and the average-case criterion [Finucane_Liu_Mitzenmacher_aller08] are two extreme cases of our objective function. A self-randomized modulation code is proposed which is asymptotically optimal, as, for an arbitrary input alphabet and i.i.d. input distribution. In practical flash memory systems, the number of cell-levels is only moderately large. So the asymptotic performance as may not tell the whole story. Using the tools from load-balancing theory, we analyze the storage efficiency of the self-randomized modulation code. The result shows that only a fraction of the cells are utilized when the number of cell-levels is only moderately large. We also propose a load-balancing modulation code, based on a phenomenon known as "the power of two random choices" [Mitzenmacher96thepower], to improve the storage efficiency of practical systems. Theoretical analysis and simulation results show that our load-balancing modulation codes can provide significant gain to practical flash memory storage systems. Though pseudo-random, our approach achieves the same load-balancing performance, for i.i.d. inputs, as a purely random approach based on the power of two random choices.

cs.IT↗

arXiv2009

Can Iterative Decoding for Erasure Correlated Sources be Universal?

In this paper, we consider a few iterative decoding schemes for the joint source-channel coding of correlated sources. Specifically, we consider the joint source-channel coding of two erasure correlated sources with transmission over different erasure channels. Our main interest is in determining whether or not various code ensembles can achieve the capacity region universally over varying channel conditions. We consider two ensembles in the class of low-density generator-matrix (LDGM) codes known as Luby-Transform (LT) codes and one ensemble of low-density parity-check (LDPC) codes. We analyze them using density evolution and show that optimized LT codes can achieve the extremal symmetric point of the capacity region. We also show that LT codes are not universal under iterative decoding for this problem because they cannot simultaneously achieve the extremal symmetric point and a corner point of the capacity region. The sub-universality of iterative decoding is characterized by studying the density evolution for LT codes.

cs.IT↗

arXiv2009

A Rate-Distortion Perspective on Multiple Decoding Attempts for Reed-Solomon Codes

Recently, a number of authors have proposed decoding schemes for Reed-Solomon (RS) codes based on multiple trials of a simple RS decoding algorithm. In this paper, we present a rate-distortion (R-D) approach to analyze these multiple-decoding algorithms for RS codes. This approach is first used to understand the asymptotic performance-versus-complexity trade-off of multiple error-and-erasure decoding of RS codes. By defining an appropriate distortion measure between an error pattern and an erasure pattern, the condition for a single error-and-erasure decoding to succeed reduces to a form where the distortion is compared to a fixed threshold. Finding the best set of erasure patterns for multiple decoding trials then turns out to be a covering problem which can be solved asymptotically by rate-distortion theory. Next, this approach is extended to analyze multiple algebraic soft-decision (ASD) decoding of RS codes. Both analytical and numerical computations of the R-D functions for the corresponding distortion measures are discussed. Simulation results show that proposed algorithms using this approach perform better than other algorithms with the same complexity.

cs.IT↗

arXiv2005

Capacity-Achieving Ensembles of Accumulate-Repeat-Accumulate Codes for the Erasure Channel with Bounded Complexity

The paper introduces ensembles of accumulate-repeat-accumulate (ARA) codes which asymptotically achieve capacity on the binary erasure channel (BEC) with {\em bounded complexity}, per information bit, of encoding and decoding. It also introduces symmetry properties which play a central role in the construction of capacity-achieving ensembles for the BEC with bounded complexity. The results here improve on the tradeoff between performance and complexity provided by previous constructions of capacity-achieving ensembles of codes defined on graphs. The superiority of ARA codes with moderate to large block length is exemplified by computer simulations which compare their performance with those of previously reported capacity-achieving ensembles of LDPC and IRA codes. The ARA codes also have the advantage of being systematic.

cs.IT↗

arXiv2005

Accumulate-Repeat-Accumulate Codes: Systematic Codes Achieving the Binary Erasure Channel Capacity with Bounded Complexity

The paper introduces ensembles of accumulate-repeat-accumulate (ARA) codes which asymptotically achieve capacity on the binary erasure channel (BEC) with {\em bounded complexity} per information bit. It also introduces symmetry properties which play a central role in the construction of capacity-achieving ensembles for the BEC. The results here improve on the tradeoff between performance and complexity provided by the first capacity-achieving ensembles of irregular repeat-accumulate (IRA) codes with bounded complexity per information bit; these IRA ensembles were previously constructed by Pfister, Sason and Urbanke. The superiority of ARA codes with moderate to large block length is exemplified by computer simulations which compare their performance with those of previously reported capacity-achieving ensembles of LDPC and IRA codes. The ARA codes also have the advantage of being systematic.

cs.IT↗