On the Fundamental Recovery Limit of Orthogonal Least Squares

Junhan Kim; Jian Wang; Byonghyo Shim

Home/Archive/arXiv

arXiv · 1910.10976

On the Fundamental Recovery Limit of Orthogonal Least Squares

Junhan Kim·Jian Wang·Byonghyo Shim

View original source ↗Read source document ↗Export metadata ↓

Abstract

Orthogonal least squares (OLS) is a classic algorithm for sparse recovery, function approximation, and subset selection. In this paper, we analyze the performance guarantee of the OLS algorithm. Specifically, we show that OLS guarantees the exact reconstruction of any $K$-sparse vector in $K$ iterations, provided that a sensing matrix has unit $\ell_{2}$-norm columns and satisfies the restricted isometry property (RIP) of order $K+1$ with \begin{align*} δ_{K+1} &<C_{K} = \begin{cases} \frac{1}{\sqrt{K}}, & K=1, \\ \frac{1}{\sqrt{K+\frac{1}{4}}}, & K=2, \\ \frac{1}{\sqrt{K+\frac{1}{16}}}, & K=3, \\ \frac{1}{\sqrt{K}}, & K \ge 4. \end{cases} \end{align*} Furthermore, we show that the proposed guarantee is optimal in the sense that if $δ_{K+1} \ge C_{K}$, then there exists a counterexample for which OLS fails the recovery.

Explore related subjects

cs.IT math.IT

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Junhan Kim, Jian Wang, Byonghyo Shim. 2020-08-21. On the Fundamental Recovery Limit of Orthogonal Least Squares. https://arxiv.org/abs/1910.10976

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Universal Feature Selection with Noisy Observations and Weak Symmetry Conditions

This paper relaxes the restrictive symmetry conditions adopted in [4], [5] and extends their universal feature selection framework to accommodate noisy observations as well as attribute structures that may exhibit directional preferences. We introduce the notion of weak spherical symmetry, quantified by second-moment distances, which allows controlled deviations from rotational invariance. Under this relaxed condition, we develop a universal feature selection framework based on the singular value decomposition of the canonical dependence matrix computed from noisy data. Our main result shows that the selected features achieve asymptotically optimal error exponents up to a residual term that depends on the symmetry deviation $δ$ and the noise levels $η_1, η_2$. When $δ, η_1, η_2$ are relatively small, our result recovers that of [5], thereby demonstrating that exact spherical symmetry is unnecessary. Overall, our findings highlight the robustness of the selection framework against second-moment deviations and observation noise, thereby broadening its applicability across diverse inference tasks and providing a theoretically grounded tool for universal feature selection in practical scenarios.

cs.IT↗

arXiv2026

CPM-LDPC Codes Attaining the Minimum-Distance Bound

We study binary quasi-cyclic LDPC codes whose parity-check matrices are full arrays of single circulant permutation matrices (CPMs), referred to here as CPM-LDPC codes. Their minimum distance is at most $(J+1)!$, where $J$ is the column weight. For every fixed pair of column and row weights $2\le J<L$, we show that this bound is attained for all sufficiently large integer lift sizes. First, we give one integer exponent matrix independent of the lift size $P$. Second, we show that independent uniform exponent choices attain the bound with probability $1-O_{J,L}(P^{-1})$. Both proofs use cycle conditions required by low-weight codewords and a lower bound on the number of terms in vectors satisfying polynomial check equations. Neither construction requires $P$ to be prime. We also give small-lift arrays attaining the bound 24 for $J=3$, $L=4,\ldots,8$, and arrays with distance at least 28 for $J=4$, $L=5,\ldots,8$, together with computational distance verification.

cs.IT↗

arXiv2026

On Sequence Reconstruction Problem for q-ary Deletion Channels

The sequence reconstruction problem for $q$-ary deletion channels, introduced by Levenshtein in 2001, concerns the minimum number of channels required to uniquely recover a transmitted sequence when each channel introduces exactly $t$ deletions. Combinatorially, it is equivalent to determining $N_q(n,d,t)$, the maximum intersection size of two $t$-deletion balls with centers at Levenshtein distance at least $d$, for $q$-ary sequences of length $n$ over the alphabet $Σ_q=\{0,1,\dots,q-1\}$. Levenshtein solved the uncoded case $N_q(n,1,t)$ for all $n\ge t$; subsequently, Gabrys and Yaakobi determined $N_2(n,2,t)$, and Wang et al. extended the result to $N_3(n,2,t)$. In this paper, we study the problem for $q$-ary sequences under minimum Levenshtein distance $d=2$ with channels that introduce exactly $t$ deletions. We determine the exact value of $N_q(n,2,t)$ for all $t\ge 2, q\geq 4$, and for sufficiently large $n$, and construct explicit pairs of sequences attaining the maximum intersection. Furthermore, for each $q\ge3$, we characterize all extremal sequence pairs. In particular, if the intersection size matches the first two terms of $N_q(n,2,t)$, then the two center sequences must contain, at the same positions, length-5 blocks of the forms $(a,b,c,a,b)$ and $(b,a,c,b,a)$ for some distinct $a,b,c\inΣ_q$; for $t\ge q+2$, the exact maximum $N_q(n,2,t)$ is attained precisely by $2q!$ unordered pairs of sequences with a specific block structure. Asymptotically, we prove that for $q\ge 4$ and $t\ge 2$, \[ N_q(n,2,t)=\frac{6}{(t-2)!}n^{t-2}-\frac{3t+13}{(t-3)!}n^{t-3}+\frac{3t^2+25t+64}{4(t-4)!}n^{t-4}+O(n^{t-5}). \] Moreover, $N_q(n,2,t)$ and $N_{q-1}(n,2,t)$ share their first $q-1$ terms, and for $t\ge q$ the coefficient of $n^{t-q}$ in their difference is $\frac{6t-6q+5}{(t-q)!}$.

cs.IT↗