arXiv Science⌕ Search

arXiv subjects

Amin Gohari

Publications and source records attributed to Amin Gohari.

At least 19 recordsLinked to original sources
arXiv2026

A Proof of the Most Informative Boolean Function Conjecture

Let $X$ be uniform on $\{-1,1\}^n$, let $Y$ be obtained by passing its coordinates independently through a binary symmetric channel with crossover probability $p$, and let $g:\{-1,1\}^n\to\{0,1\}$ be a Boolean function. We give a computer-assisted proof of the Courtade--Kumar conjecture $I(g(X);Y)\le1-H_2(p)$, where $H_2$ is binary entropy, with equality attained by dictator functions. The present work builds on the differential-equation method, itself a limiting form of the auxiliary-receiver approach in network information theory using a continuum of degraded receivers. The proof proceeds from a local inequality to a dimension-independent bound on entropy production. Differentiation along the Boolean noise semigroup expresses entropy production as an average of edge costs. The key estimate is therefore an unrestricted Bellman inequality with two mean constraints and two entropy constraints, allowing arbitrary couplings of the edge variables. This paper and its supplement provide the proofs and computational verification records. The document is lengthy because it is designed to be entirely self-contained, deriving all proofs from first principles and reproducing the proofs of cited results. The supplementary material supporting the computer-assisted parts of the proof are available online.

cs.DS↗
arXiv2026

On Dependence Measures Based on $Φ$-Divergence, $Φ$-Entropy, and Their Matrix Forms

We study two information measures associated with a convex function $Φ$: the $Φ$-mutual information $\IPhi$ and the max-$Φ$-mutual information $\wIPhi$. The measure $I_Φ$ is naturally induced by $Φ$-divergence and $Φ$-entropy and admits a chain-rule-compatible conditional form, but it lacks several Shannon-type identities. To address this, we introduce and analyze $\wIPhi$, a variational dependence measure obtained by optimizing over auxiliary random variables from $\IPhi$. We establish basic calculus rules for both quantities, including chain-rule identities, data-processing-type inequalities, deterministic function rules and Markov-chain characterizations. For suitable classes of $Φ$, we obtain closed-form expressions for $\wIPhi$ and identify conditions under which KL divergence is the unique case with full Shannon-type behavior. We extend the above notions to the matrix case, with the existing definitions of the matrix $Φ$-entropy and the matrix $Φ$-mutual information. We propose the matrix $Φ$-ribbons, prove tensorization and data processing properties. As applications, we derive monotonicity of matrix $Φ$-ribbons for wirings of no-signaling boxes, study associated SDPI constants, and obtain ribbon regions from partial independence structures and non-Shannon-type information inequalities.

cs.IT↗
arXiv2026

Exact Rate Exponent Tradeoff for New Classes of Distributed Hypothesis Testing Problems

We characterize the exact rate--exponent tradeoff for new classes of one-way distributed hypothesis testing problems by demonstrating that a recent upper bound, derived via the auxiliary-receiver technique, coincides with known lower bounds. We achieve this by relaxing the upper bound on the type-II error exponent into a form that shares the same inner functional as Han's lower bound, differing only in the outer rate constraint. Furthermore, we prove that this upper bound is tight for testing against dependence and for the doubly symmetric binary source (DSBS) with crossover probabilities $κ_0$ under the null hypothesis and $κ_1$ under the alternative hypothesis, provided $0 < κ_1 < κ_0 < \frac12$. The characterization of the exact error exponent for the DSBS source holds for every communication rate $R \geq 0$.

cs.IT↗
arXiv2026

Imaginative Generative AI: Crossing the Entropy Wall into Worlds Beyond Imitation

Generative AI models are primarily designed to imitate the data distribution, an objective that neither corrects diversity lost by a learned generator nor defines how generation should extend beyond the diversity of the data itself. We introduce Imaginative Generative AI (IGA), a framework that makes diversity part of the target-distribution design problem: among distributions close to a reference, IGA selects one whose spectral diversity reaches a prescribed level. Diversity is measured by the von Neumann entropy of the generated distribution's kernel covariance operator in a fixed representation space, providing a reference-free representation-guided measure of how broadly probability mass occupies embedding directions. The spectral entropy of the population data distribution defines an Entropy Wall. Below the wall, IGA performs diversity repair, recovering variation that a learned generator has lost while remaining within the diversity level of the data. Beyond the wall, the data distribution itself becomes infeasible, and IGA deliberately departs from it to produce distributions with greater representation-relative spectral diversity, an operational notion of imaginative generation. These regimes form a single regularization path from imitation to imagination and define an i.i.d. target distribution at each prescribed diversity level. We develop the theory of this entropy-constrained projection and show that, under a KL anchor to a pretrained generator, the optimum satisfies a self-consistent exponential-tilt relation. This characterization leads to IGA Guidance, a retraining-free inference-time method for score-based and diffusion models, including DDPM and DDIM samplers. Experiments on synthetic and vision benchmarks demonstrate diversity repair below the Entropy Wall and controlled spectral extrapolation beyond it.

cs.LG↗
arXiv2026

A Differential Equation Approach to the Most-Informative Boolean Function Conjecture

We study the most-informative Boolean function conjecture using a differential equation approach. This leads to a formulation of a functional inequality on finite-dimensional random variables. We also develop a similar inequality in the case of the Hellinger conjecture. Finally, we conjecture a specific finite dimensional inequality that, if proved, will lead to a proof of the Boolean function conjecture in the balanced case. We further show that the above inequality holds modulo four explicit inequalities (all of which seems to hold via numerical simulation) with the first three containing just two variables and a final one involving four variables.

cs.IT↗
arXiv2026

The Capacity Region for Classes of Sum-Broadcast Channels

We compute the capacity region of a sum of broadcast channels whose components are degraded, less-noisy, more-capable, deterministic, or semi-deterministic. We achieve this by showing that an auxiliary-receiver outer bound, previously introduced by some of the authors, matches Marton's inner bound. This result generalizes a previously known result for the sum of two reversely degraded broadcast channels due to El Gamal (1980). Moreover, we define a class of primary broadcast channels and show an analogous result for the sum of primary broadcast channels.

cs.IT↗
arXiv2026

Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs

Generative models guided by text prompts are widely evaluated for fidelity and prompt alignment, yet their ability to produce outputs remains underexplored. Existing diversity metrics such as Vendi and RKE, which are based on the von Neumann and Rényi entropies of kernel matrices, were developed for unconditional models and cannot distinguish prompt-induced from model-induced variability. We address this gap by introducing \textit{Conditional-Vendi} and \textit{Conditional-RKE}, diversity measures derived from the conditional entropy of positive semidefinite matrices. These scores isolate model-induced diversity in prompt-guided generation, with Conditional-RKE enjoying an $O(1/\sqrt{n})$ convergence rate. For Conditional-Vendi, we introduce a truncated-spectrum approximation that yields scalable and consistent estimates. Experiments on text-to-image, image-captioning, and LLM tasks show that the conditional scores recover ground-truth diversity orderings and can also guide diffusion models toward more diverse samples. The codebase is available at https://github.com/mjalali/conditional-vendi.

cs.LG↗
arXiv2026

Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search

Proving Shannon-type entropy inequalities is a fundamental task in information theory that often requires constructing non-trivial linear combinations of known constraints, which is a combinatorial search problem that scales poorly with the number of random variables. We investigate whether small-scale large language models (0.6B--1.7B parameters), fine-tuned on atomic proof steps and combined with guided beam search, can automate this process. On a held-out test set of 60 inequalities spanning n=10 to 15 variables, our 0.6B fine-tuned model achieves an 85\% proof success rate with tree search. GPT-5.5 solves 1.7\% samples under zero-shot prompting while Psitip solves 33.3\% samples. A systematic ablation study across training context length (4096 vs.\ 8192 tokens) and data distribution (n=9-skewed vs not skewed) reveals that a 4096-token not skewed training distribution yields the best performance, with extended context and skewed data providing no marginal benefit. We further identify two dominant failure modes -- format failures and step quality degradation -- and verify that the beam-scoring heuristic is essential via a controlled ablation (random scoring reduces success from 83\% to 23\%).

cs.IT↗
arXiv2026

Dependence Balance and Capacity Bounds for Multiterminal Communication and Wiretap Channels

An information measure based on fractional partitions of a set is used to derive a general dependence balance inequality for communication. This inequality is used to obtain new upper bounds on reliable and secret rates for multiterminal channels. For example, we obtain a new upper bound on the rate of shared randomness generated among terminals, a counterpart of the cut-set bound for reliable communication. The bounds for reliable communication use the concept of auxiliary receivers, and we show that they are optimized by Gaussian distributions for Gaussian channels. The bounds are applied to multiaccess channels with generalized feedback and relay channels, and improve the cut-set bound for scalar Gaussian channels. The improvement for Gaussian relay channels complements results obtained with other methods.

cs.IT↗
arXiv2026

The capacity region of classes of product broadcast channels

We establish a new outer bound for the capacity region of product broadcast channels. This outer bound matches Marton's inner bound for a variety of classes of product broadcast channels whose capacity regions were previously unknown. These classes include product of reversely semi-deterministic and product of reversely more-capable channels. A significant consequence of this new outer bound is that it establishes, via an example, that the previously best known outer-bound is strictly suboptimal for the general broadcast channel. Our example is comprised of a product broadcast channel with two semi-deterministic components in reverse orientation.

cs.IT↗
arXiv2026

List Estimation

Classical estimation outputs a single point estimate of an unknown $d$-dimensional vector from an observation. In this paper, we study \emph{$k$-list estimation}, in which a single observation is used to produce a list of $k$ candidate estimates and performance is measured by the expected squared distance from the true vector to the closest candidate. We compare this centralized setting with a symmetric decentralized MMSE benchmark in which $k$ agents observe conditionally i.i.d.\ measurements and each agent outputs its own MMSE estimate. On the centralized side, we show that optimal $k$-list estimation is equivalent to fixed-rate $k$-point vector quantization of the posterior distribution and, under standard regularity conditions, admits an exact high-rate asymptotic expansion with explicit constants and decay rate $k^{-2/d}$. On the decentralized side, we derive lower bounds in terms of the small-ball behavior of the single-agent MMSE error; in particular, when the conditional error density is bounded near the origin, the benchmark distortion cannot decay faster than order $k^{-2/d}$. We further show that if the error density vanishes at the origin, then the decentralized benchmark is provably unable to match the centralized $k^{-2/d}$ exponent, whereas the centralized estimator retains that scaling. Gaussian specializations yield explicit formulas and numerical experiments corroborate the predicted asymptotic behavior. Overall, the results show that, in the scaling with $k$, one observation combined with $k$ carefully chosen candidates can be asymptotically as effective as -- and in some regimes strictly better than -- this MMSE-based decentralized benchmark with $k$ independent observations.

cs.IT↗
arXiv2026

Source Coding with Free Bits and the Multi-Way Number Partitioning Problem

We introduce a new variant of variable-length source coding for sending a source over two parallel channels, one of which is costly and the other free. We give a complete solution to this problem. Next, we relate the problem to the number partitioning problem, which is the task of dividing a given list of numbers into a pre-specified number of subsets such that the sum of the numbers in each subset is as nearly equal as possible. We introduce two new objective functions for this problem and show that an adapted version of the Huffman coding algorithm (with a runtime of $\mathcal{O}(n \log n)$ for input size $n$) produces the optimal solution for one objective function, and a nearly optimal solution for the other objective function.

cs.DS↗
arXiv2026

Subjective Distortion: Achievability and Outer Bounds for Distortion Functions with Memory

In some rate-distortion-type problems, the required fidelity of information is affected by past actions. As a result, the distortion function depends not only on the instantaneous distortion between a source symbol and its representation symbol, but also on past representations. In this paper, we give a formal definition of this problem and introduce both inner (achievable) and outer bounds on the rate-distortion tradeoff. We also discuss convexification of the problem, which makes it easier to find bounds. Problems of this type arise in biological information processing, as well as in recommendation engines; we provide an example applied to a simplified biological information processing problem.

cs.IT↗
arXiv2026

Ribbons from Independence Structure: Hypercontractivity, $Φ$-Mutual Information, and Matrix $Φ$-Entropy

We study the hypercontractivity ribbon and the $Φ$-ribbon for joint distributions that obey a given independence structure, obtaining tight bounds in some basic regimes. For general independence structures, modeled as a hypergraph whose hyperedges specify mutually independent subcollections of random variables, we provide an explicit inner bound on the $Φ$-ribbon described by a simple convex hull of incidence vectors. We also provide a new multipartite generalization version and a $Φ$-mutual information analogue of the Zhang--Yeung inequality, which implies nontrivial points in the hypercontractivity ribbon and the $Φ$-ribbon respectively. Finally, we propose the matrix $Φ$-ribbon based on matrix $Φ$-entropy and establish the tensorization and data processing properties, together with the calculation of an exact matrix SDPI constant for the doubly symmetric binary source.

cs.IT↗
arXiv2026

Stable Source Coding

A source encoder is stable if a small change in the source sequence (e.g., changing a few symbols) results in a small (or bounded) change in the output codeword. By this definition, the common technique of random binning is unstable; because the mapping is random, two nearly identical source sequences can be assigned to completely unrelated bin indices. We study compression rates of stable lossless source codes. Using combinatorial arguments, we derive information-theoretic limits on the achievable rate as a function of the stability parameters.

cs.IT↗
arXiv2026

A Class of Subadditive Information Measures and their Applications

We introduce a two-parameter family of discrepancy measures, termed \emph{$(G,f)$-divergences}, obtained by applying a non-decreasing function $G$ to an $f$-divergence $D_f$. Building on Csiszár's formulation of mutual $f$-information, we define a corresponding $(G,f)$-information measure $ I_{G,f}(X;Y)$. A central theme of the paper is subadditivity over product distributions and product channels. We develop reduction principles showing that, for broad classes of $G$, it suffices to verify divergence subadditivity on binary alphabets. Specializing to the functions $G(x)\in\{x,\log(1+x),-\log(1-x)\}$, we derive tractable sufficient conditions on $f$ that guarantee subadditivity, covering many standard $f$-divergences. Finally, we present applications to finite-blocklength converses for channel coding, bounds in binary hypothesis testing, and an extension of the Shannon--Gallager--Berlekamp sphere-packing exponent framework to subadditive $(G,f)$-divergences.

cs.IT↗
arXiv2025

SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score

Diffusion models have demonstrated remarkable success in high-fidelity image synthesis and prompt-guided generative modeling. However, ensuring adequate diversity in generated samples of prompt-guided diffusion models remains a challenge, particularly when the prompts span a broad semantic spectrum and the diversity of generated data needs to be evaluated in a prompt-aware fashion across semantically similar prompts. Recent methods have introduced guidance via diversity measures to encourage more varied generations. In this work, we extend the diversity measure-based approaches by proposing the Scalable Prompt-Aware Rény Kernel Entropy Diversity Guidance (SPARKE) method for prompt-aware diversity guidance. SPARKE utilizes conditional entropy for diversity guidance, which dynamically conditions diversity measurement on similar prompts and enables prompt-aware diversity control. While the entropy-based guidance approach enhances prompt-aware diversity, its reliance on the matrix-based entropy scores poses computational challenges in large-scale generation settings. To address this, we focus on the special case of Conditional latent RKE Score Guidance, reducing entropy computation and gradient-based optimization complexity from the $O(n^3)$ of general entropy measures to $O(n)$. The reduced computational complexity allows for diversity-guided sampling over potentially thousands of generation rounds on different prompts. We numerically test the SPARKE method on several text-to-image diffusion models, demonstrating that the proposed method improves the prompt-aware diversity of the generated data without incurring significant computational costs. We release our code on the project page: https://mjalali.github.io/SPARKE

cs.CV↗
arXiv2025

On the Source Model Key Agreement Problem

We consider the source model key agreement problem involving two legitimate parties and an eavesdropper who observe n i.i.d. samples of X, Y, and Z, respectively. In this paper, we focus on one of the simplest instances where the key capacity remains open, specifically when X and Y are binary random variables and Z is a function of the pair (X, Y). The best-known upper bound on the key capacity is characterized by an inf-max optimization problem that generally lacks a closed-form solution. We provide general conditions under which the upper bound reduces to I(X;Y). As an example, we consider the XOR setting in which X and Y are binary, and Z is the XOR of X and Y. The upper bound reduces to I(X;Y) for this source. Next, we conjecture that the rate I(X;Y) is not achievable for the XOR source and provide some ideas that might be useful for developing a new upper bound on the source model problem.

cs.IT↗