arXiv subjects

Yanxiao Liu

Publications and source records attributed to Yanxiao Liu.

At least 19 recordsLinked to original sources

arXiv2026

Watermarkable Multi-Draft Speculative Sampling via Poisson Processes

Large language models (LLMs) have achieved state-of-the-art performance across a wide range of tasks, motivating two important aspects of deployment: inference efficiency and output provenance, which can be tackled by speculative sampling and watermarking, respectively. However, recent works have shown that combining these two goals is highly nontrivial and can be potentially impossible. In this work, we develop a novel multi-draft speculative sampling algorithm based on Poisson processes that improves the frontier of this fundamental trade-off. The proposed algorithm has strong sampling efficiency on its own and, more interestingly, is naturally watermarkable: we can embed an unbiased watermark without degrading speculative acceptance. Moreover, our algorithm is based on an exact list-coupling-without-communication scheme, which yields a drafter invariance property that benefits both sampling and watermarking. It is the first multi-draft, drafter-invariant speculative sampling scheme that maintains both watermark strength and sampling efficiency, and we experimentally verify its strong performance in both aspects.

cs.CR↗

arXiv2026

ExpBoN: Exponential-Noise Best-of-$n$ for Efficient Test-Time LLM Alignment

Best-of-$n$ (BoN) sampling is a simple yet effective inference-time alignment method, but hard maximization provides only coarse control over the trade-off between reward and distribution shift. Soft Best-of-$n$ (Verdun et al. 2025) provides smoother control and converges to the optimal distribution associated with KL-regularized reward maximization. In this paper, we introduce ExpBoN, an alternative soft BoN method based on the exponential-noise report-noisy-max mechanism. It admits an exact finite-$n$ decomposition, which yields exponentially fast convergence in total variation, expected reward, and both directions of KL divergence. We provide comprehensive theoretical analyses of its convergence and regret behavior. We further integrate ExpBoN into the guided speculative inference (GSI) framework (Geuter, Mroueh, and AlvarezMelis 2025), resulting in ExpGSI, for efficient reward-guided LLM alignment. ExpGSI yields substantial reductions in computational cost while maintaining comparable accuracy. Experiments on MATH500, MMLU-STEM, and Minerva Math with the Qwen2.5-Math and Qwen3 model families show that ExpGSI reduces estimated computation by $14\%$-$39\%$ across candidate budgets for Qwen2.5-Math and by up to $45\%$ at $n=16$ for Qwen3. Overall, our results provide a theoretical and algorithmic foundation for exponential-noise BoN and efficient test-time LLM alignment.

cs.LG↗

arXiv2026

One-Shot Information Hiding and Compound Wiretap Channels

In recent years, due to the growing reliance on large amounts of data that are communicated, analyzed, and utilized, which inherently contain sensitive and personal information, secrecy and privacy in communication have become increasingly important. We present nonasymptotic information-theoretic analyses of two fundamental secrecy problems under channel uncertainties: the information hiding problem and the compound wiretap channel. The former admits a game-theoretic formulation, where one party (an information hider and a decoder) seeks to embed secret messages into a host signal for later reconstruction, while the opposing party (an attacker) attempts to remove or degrade the embedded information. The latter generalizes Wyner's wiretap channel by allowing multiple potential channel states. The information hiding problem concerns active attacks during data transmission, while the compound wiretap channel addresses passive eavesdropping and information leakage. The two problems, both of which consider channels with uncertainties, can be studied under a unified framework by utilizing a covering argument and the Poisson matching lemma by Li and Anantharam. We derive novel one-shot achievability results for both problems that are applicable to any source distribution and any class of channels (not necessarily memoryless or ergodic), and that apply to both discrete and continuous cases. We also show that existing asymptotic results on both problems can be recovered by applying our results to discrete memoryless channels.

cs.IT↗

arXiv2026

Accelerating Diffusion Sampling via Speculative Draft Trees

Speculative sampling accelerates diffusion model generation by drafting inexpensive candidate states and correcting them under a coupling that preserves the target distribution exactly, reducing the number of expensive target evaluations. Existing diffusion samplers, notably those based on reflection maximal coupling, are topologically constrained: their lookahead drafts form a chain graph, a single linear sequence, which inherently limits the acceptance rate per target evaluation. We connect speculative sampling in diffusion models to relative entropy coding (REC). This perspective shows the lookahead need not be linear and motivates our central contribution, draft trees, which enrich the candidates considered per round and lower the target function evaluations. We further adopt greedy rejection sampling, an REC algorithm, as the draft-target coupling, improving acceptance while guaranteeing exact target samples. Experiments across diverse target and draft models demonstrate up to 8.3% acceleration over the reflection coupling baseline in practical settings.

cs.LG↗

arXiv2026

Sub-optimality of Marton's Inner Bound for the Two-Receiver Broadcast Channel

Marton's inner bound, the best-known achievable region for a general discrete memoryless broadcast channel, was proposed by Katalin Marton in 1979, and whether it always achieves the capacity region has remained open since then. In this paper, we establish its strict sub-optimality: we show that the capacity region of some discrete memoryless broadcast channels can be strictly larger than Marton's inner bound.

cs.IT↗

arXiv2026

Counterexamples to the Markovity Conjecture for the Two-Receiver Broadcast Channel

We present two counterexamples to the \emph{Markovity Conjecture} of Gohari, Liu and Nair (ISIT 2025), which is a structural conjecture concerning the optimizers of the dual functional associated with Marton's inner bound and, if true, would greatly simplify the evaluation of Marton's inner bound. Both counterexamples are ternary-input broadcast channels with strictly positive transition probabilities and use the same nonrectangular $2\times2$ auxiliary structure. In each case, we exhibit an explicit non-Markov construction whose objective value is rigorously larger than that achievable by any construction satisfying the conjectured Markov structure. Both examples are obtained with the assistance of GPT-5.6 Sol and disprove the Markovity Conjecture.

cs.IT↗

arXiv2026

Scalable Differentially Private Data Compression via Diffusion and Stochastic Codes

The ever-increasing collection of personal data has created mounting pressure to develop technologies that protect sensitive aspects of individual identity. Differential privacy (DP) provides a principled framework with strong formal guarantees and has already achieved practical success. However, releasing high-dimensional data, such as images, has remained elusive: releasing uncompressed privatized data requires significant storage. At the same time, no effective data compression scheme exists that can compress high-resolution data with privacy guarantees. We address this challenge with DP-DiPP, a compression pipeline that combines stochastic codes with diffusion models. DP-DiPP is highly flexible: the practitioner has direct control over the compression rate-privacy-utility tradeoff. As the theoretical backbone, we extend the Poisson private representation (PPR) to encode the outputs of privacy mechanisms. We then combine it with DiffC, a diffusion-based lossy data compression method, to obtain a differentially private image compressor. Our experiments on privatized image classification on CIFAR-10 demonstrate that DP-DiPP significantly outperforms the baseline, achieving a 10-30 times better compression while retaining comparable privacy guarantees and utility.

cs.CR↗

arXiv2026

Tighter Information-Theoretic Generalization Bounds via a Novel Class of Change of Measure Inequalities

Change of measure inequalities translate divergences between probability measures into explicit bounds on event probabilities, and play an important role in deriving probabilistic guarantees in learning theory, information theory, and statistics. We propose novel change of measure inequalities via a unified framework based on the data processing inequality, which is surprisingly elementary yet powerful enough to yield novel, tighter inequalities. We provide change of measure inequalities in terms of a broad family of information measures, including $f$-divergences (with Kullback-Leibler divergence and $χ^2$-divergence as special cases), Rényi divergence, and $α$-mutual information (with maximal leakage as a special case). We apply these results to generalization error analysis, PAC-Bayesian theory, differential privacy, and data memorization, obtaining stronger guarantees while recovering best-known results through simplified analyses.

cs.IT↗

arXiv2026

Nonasymptotic Oblivious Relaying and Variable-Length Noisy Lossy Source Coding

The information bottleneck channel (or the oblivious relay channel) concerns a channel coding setting where the decoder does not directly observe the channel output. Rather, the channel output is relayed to the decoder by an oblivious relay (which does not know the codebook) via a rate-limited link. The capacity is known to be given by the information bottleneck. We study finite-blocklength achievability results of the channel, where the relay communicates to the decoder via fixed-length or variable-length codes. These two cases give rise to two different second-order versions of the information bottleneck. Our proofs utilize the nonasymptotic noisy lossy source coding results by Kostina and Verdú, the strong functional representation lemma, and the Poisson matching lemma. Moreover, we also give a novel nonasymptotic variable-length noisy lossy source coding result.

cs.IT↗

arXiv2026

On the Generalization Error of Differentially Private Algorithms via Typicality

We study the generalization error of stochastic learning algorithms from an information-theoretic perspective, with a particular emphasis on deriving sharper bounds for differentially private algorithms. It is well known that the generalization error of stochastic learning algorithms can be bounded in terms of mutual information and maximal leakage, yielding in-expectation and high-probability guarantees, respectively. In this work, we further upper bound mutual information and maximal leakage by explicit, easily computable formulas, using typicality-based arguments and exploiting the stability properties of private algorithms. In the first part of the paper, we strictly improve the mutual-information bounds by Rodríguez-Gálvez et al. (IEEE Trans. Inf. Theory, 2021). In the second part, we derive new upper bounds on the maximal leakage of learning algorithms. In both cases, the resulting bounds on information measures translate directly into generalization error guarantees.

cs.IT↗

arXiv2026

Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unification

"Compression Tells Intelligence", is supported by research in artificial intelligence, particularly concerning (multimodal) large language models (LLMs/MLLMs), where compression efficiency often correlates with improved model performance and capabilities. For compression, classical visual coding based on traditional information theory has developed over decades, achieving great success with numerous international industrial standards widely applied in multimedia (e.g., image/video) systems. Except that, the recent emergingvisual token technology of generative multi-modal large models also shares a similar fundamental objective like visual coding: maximizing semantic information fidelity during the representation learning while minimizing computational cost. Therefore, this paper provides a comprehensive overview of two dominant technique families first -- Visual Coding and Vision Token Technology -- then we further unify them from the aspect of optimization, discussing the essence of compression efficiency and model performance trade-off behind. Next, based on the proposed unified formulation bridging visual coding andvisual token technology, we synthesize bidirectional insights of themselves and forecast the next-gen visual codec and token techniques. Last but not least, we experimentally show a large potential of the task-oriented token developments in the more practical tasks like multimodal LLMs (MLLMs), AI-generated content (AIGC), and embodied AI, as well as shedding light on the future possibility of standardizing a general token technology like the traditional codecs (e.g., H.264/265) with high efficiency for a wide range of intelligent tasks in a unified and effective manner.

cs.CV↗

arXiv2025

One-Shot Coding and Applications

One-shot information theory addresses scenarios in source coding and channel coding where the signal blocklength is assumed to be 1. In this case, each source and channel can be used only once, and the sources and channels are arbitrary and not required to be memoryless or ergodic. We study the achievability part of one-shot information theory, i.e., we consider explicit coding schemes in the oneshot scenario. The objective is to derive one-shot achievability results that can imply existing (first-order and second-order) asymptotic results when applied to memoryless sources and channels, or applied to systems with memory that behave ergodically. Poisson functional representation was first proposed as a one-shot channel simulation technique by Li and El Gamal [118] for proving a strong functional representation lemma. It was later extended to the Poisson matching lemma by Li and Anantharam [117], which provided a unified one-shot coding scheme for a broad class of information-theoretic problems. The main contribution of this thesis is to extend the applicability of Poisson functional representation to various more complicated scenarios, where the original version cannot be applied directly and further extensions must be developed.

cs.IT↗

arXiv2025

Joint Scheduling and Multiflow Maximization in Wireless Networks

Towards the development of 6G mobile networks, it is promising to integrate a large number of devices from multi-dimensional platforms, and it is crucial to have a solid understanding of the theoretical limits of large-scale networks. We revisit a fundamental problem at the heart of network communication theory: the maximum multiflow (MMF) problem in multi-hop networks, with network coding performed at intermediate nodes. To derive the exact-optimal solution to the MMF problem (as opposed to approximations), conventional methods usually involve two steps: first calculate the scheduling rate region, and then find the maximum multiflow that can be supported by the achievable link rates. However, the NP-hardness of the scheduling part makes solving the MMF problem in large networks computationally prohibitive. In this paper, while still focusing on the exact-optimal solution, we provide efficient algorithms that can jointly calculate the scheduling rate region and solve the MMF problem, thereby outputting optimal values without requiring the entire scheduling rate region. We theoretically prove that our algorithms always output optimal solutions in a finite number of iterations, and we use various simulation results to demonstrate our advantages over conventional approaches. Our framework is applicable to the most general scenario in multi-source multi-sink networks: the multiple multicast problem with network coding. Moreover, by employing a graphical framework, we show that our algorithm can be extended to scenarios where propagation delays are large (e.g., underwater networks), in which recent studies have shown that the scheduling rate region can be significantly improved by utilizing such delays.

cs.IT↗

arXiv2025

One-Shot Coding over General Noisy Networks

We present a unified one-shot coding framework designed for the communication and compression of messages among multiple nodes across a general acyclic noisy network. Our setting can be seen as a one-shot version of the acyclic discrete memoryless network studied by Lee and Chung, and noisy network coding studied by Lim, Kim, El Gamal and Chung. We design a proof technique, called the exponential process refinement lemma, that is rooted in the Poisson matching lemma by Li and Anantharam, and can significantly simplify the analyses of one-shot coding over multi-hop networks. Our one-shot coding theorem not only recovers a wide range of existing asymptotic results, but also yields novel one-shot achievability results in different multi-hop network information theory problems, such as compress-and-forward and partial-decode-and-forward bounds for a one-shot (primitive) relay channel, and a bound for one-shot cascade multiterminal source coding. In a broader context, our framework provides a unified one-shot bound applicable to any combination of source coding, channel coding and coding for computing problems.

cs.IT↗

arXiv2024

Universal Exact Compression of Differentially Private Mechanisms

To reduce the communication cost of differential privacy mechanisms, we introduce a novel construction, called Poisson private representation (PPR), designed to compress and simulate any local randomizer while ensuring local differential privacy. Unlike previous simulation-based local differential privacy mechanisms, PPR exactly preserves the joint distribution of the data and the output of the original local randomizer. Hence, the PPR-compressed privacy mechanism retains all desirable statistical properties of the original privacy mechanism such as unbiasedness and Gaussianity. Moreover, PPR achieves a compression size within a logarithmic gap from the theoretical lower bound. Using the PPR, we give a new order-wise trade-off between communication, accuracy, central and local differential privacy for distributed mean estimation. Experiment results on distributed mean estimation show that PPR consistently gives a better trade-off between communication, accuracy and central differential privacy compared to the coordinate subsampled Gaussian mechanism, while also providing local differential privacy.

cs.CR↗

arXiv2023

Wireless Network Scheduling with Discrete Propagation Delays: Theorems and Algorithms

This paper focuses on the link scheduling problem in networks where signal delays between nodes are multiples of a time interval. To model such networks, a directed hypergraph is employed, along with an integer matrix that specifies the delays. The link scheduling problem is closely connected to the independent sets of the periodic hypergraph induced by the network model. However, due to the infinite number of vertices, it is impractical to enumerate the independent sets of the periodic hypergraph using generic graph algorithms. To tackle this challenge, a graphical approach is proposed in this paper. The link scheduling rate region is characterized using a finite directed graph called a scheduling graph, which is derived from the network model. A collision-free schedule of the network corresponds to a path in the scheduling graph, and the rate region is determined by the convex hull of the rate vectors associated with the cycles in the scheduling graph. Although existing cycle enumeration algorithms can be employed to calculate the rate region, their computational complexity becomes prohibitively high as the size of the scheduling graph grows exponentially with the number of network links. To address this issue, the dominance property of a special scheduling graph called the step-T scheduling graph is investigated. This property allows the utilization of specific subgraphs of the step-T scheduling graph to characterize the scheduling rate region, achieving a reduction in both the number of cycles and their lengths. For common problems such as calculating the rate region and maximizing a weighted sum of the scheduling rates, algorithms leveraging the dominance property are developed. These algorithms can be more efficient than using generic graph algorithms directly on the scheduling graphs.

cs.IT↗

arXiv2023

Weighted Parity-Check Codes for Channels with State and Asymmetric Channels

In this paper, we introduce a new class of codes, called weighted parity-check codes, where each parity-check bit has a weight that indicates its likelihood to be one (instead of fixing each parity-check bit to be zero). It is applicable to a wide range of settings, e.g. asymmetric channels, channels with state and/or cost constraints, and the Wyner-Ziv problem, and can provably achieve the capacity. For the channels with state (Gelfand-Pinsker) setting, the proposed coding scheme has two advantages compared to the nested linear code. First, it achieves the capacity of any channel with state (e.g. asymmetric channels). Second, simulation results show that the proposed code achieves a smaller error rate compared to the nested linear code. We also discuss a sparse construction where the belief propagation algorithm can be applied to improve the coding efficiency.

cs.IT↗

arXiv2023

Reliable Throughput of Generalized Collision Channel without Synchronization

We consider a generalized collision channel model for general multi-user communication systems, an extension of Massey and Mathys' collision channel without feedback for multiple access communications. In our model, there are multiple transmitters and receivers sharing the same communication channel. The transmitters are not synchronized and arbitrary time offsets between transmitters and receivers are assumed. A ``collision" occurs if two or more packets from different transmitters partially or completely overlap at a receiver. Our model includes the original collision channel as a special case. This paper focuses on reliable throughputs that are approachable for arbitrary time offsets. We consider both slot-synchronized and non-synchronized cases and characterize their reliable throughput regions for the generalized collision channel model. These two regions are proven to coincide. Moreover, it is shown that the protocol sequences constructed for multiple access communication remain ``throughput optimal" in the generalized collision channel model. We also identify the protocol sequences that can approach the outer boundary of the reliable throughput region.

cs.IT↗