arXiv subjects

Hyeji Kim

Publications and source records attributed to Hyeji Kim.

At least 19 recordsLinked to original sources

arXiv2026

Learned Diffusion Channel Priors for Coded MIMO Turbo Receivers

Iterative receivers for coded multiple-input multiple-output (MIMO) links estimate the channel from pilots, decoded data, and a prior. In short packets with few pilots and at low signal-to-noise ratio (SNR), the prior matters. An expectation propagation (EP) turbo receiver needs the channel's mean and covariance given an anisotropic Gaussian message from the pilots and decoder. A diffusion prior's denoiser, however, returns only a mean, and only for a channel in isotropic noise. We replace only the receiver's channel-prior module with a learned diffusion prior. At an isotropic query formed from the message, one network gives both moments: its output is the mean, its Jacobian the covariance. They form a matrix-valued EP site, the Gaussian factor representing the prior. We show why an indefinite Jacobian can make the receiver diverge, and we floor its eigenvalues. In coded block error rate (BLER), the diffusion prior fails less than a Gaussian mixture model (GMM) prior learned from the same channels. It does so on a controlled sparse specular model and a 3GPP model, at every tested receive-array and training-set size. On the former, scalar-site controls, with one variance instead of the covariance, keep this advantage, so the gain comes from the diffusion prior.

eess.SP↗

arXiv2026

Vision Token Pruning via Query-Vision Interaction Decomposition

Large vision-language models (VLMs), which process both visual inputs and text queries, incur high inference costs due to the large number of visual tokens, making visual token pruning a natural approach for improving efficiency. Recent work has leveraged query information to guide token selection, often through heuristics derived from attention patterns or token dynamics. However, these approaches do not explicitly exploit the underlying structure of query-vision interactions. We address this gap by observing that the query-vision interaction matrix, naturally induced by the query and key projections already present in transformer attention, is effectively low-rank, with a small number of dominant latent interaction modes capturing query-relevant semantics. Motivated by this observation, we propose Query-Vision Decomposition (QViD), a novel training-free query-aware visual token pruning method that exploits this structure. Extensive experiments on both image and video understanding benchmarks show that QViD consistently improves performance under matched token budgets, with the largest gains appearing under aggressive compression regimes.

cs.LG↗

arXiv2026

Grid-Supervised Flow Refinement for Wireless Localization

This paper introduces FlowLoc, a framework for high-accuracy user equipment positioning from channel state information observed at multiple base stations. The input is a magnitude channel impulse response fingerprint derived from channel estimates. FlowLoc combines a learned initial position estimate with a flow model that corrects the remaining position error. First, FlowLoc trains base station encoders using grid supervision and then removes the grid classifiers. Second, it freezes the encoders and trains a network to estimate an initial position, called the anchor, from their outputs. Third, it freezes the anchor network and trains a flow model to correct the remaining position error. We evaluate FlowLoc using ray-traced channels from an outdoor urban region, with separate trajectories for training and testing. FlowLoc reduces mean localization error from 0.917m for a grid-based localization baseline adapted from prior work to 0.537m, a 41.4% reduction. It also outperforms supervised regression and direct-coordinate generative baselines. FlowLoc retains sub-meter mean error on independently sampled channels across mobility settings from pedestrian speeds to 25m/s. Further experiments quantify sensitivity to channel noise and missing base station observations.

cs.NI↗

arXiv2026

Structured Masked Diffusion for Shared-Codebook Unsourced Random Access Decoding

In shared-codebook unsourced random access (URA), a receiver must recover an unordered set of messages from simultaneous transmissions. Its outer decoder assembles ambiguous slot-wise evidence into distinct codewords while enforcing parity constraints. We propose CIDER, a learned outer decoder that combines masked-diffusion refinement with cross-row demixing and parity-aware propagation on the code's Tanner graph. At higher per-bin loads, quality-guided remasking selectively re-decodes low-confidence rows. On the evaluated GF(64) LDPC codes at $K=2$, CIDER has lower measured symbol and codeword error rates than SIC-BP and FFT-BP at all four code lengths. In same-GPU decoder-only measurements, the deployed mixed-precision implementation is $8.2\times$ to $167\times$ faster than our optimized FFT-BP implementation across those lengths. We also evaluate higher loads, estimated user counts, and detector mismatch. Code is available at https://github.com/jiyunyoung/CIDER/tree/neurips-2026.

cs.IT↗

arXiv2026

Minimum Rate For Partially Observable Linear System with Side Information: LQG Plant and Gaussian-Markov Source

This paper studies the minimum rate required for a partially observable linear system with side information. The Linear Quadratic Gaussian(LQG) plant and the Gaussian-Markov source are considered. We show that a class of linear policies is sufficient for optimizing the conditional directed information lower bound. We also show that the resulting optimization problem is convex for the scalar case in both time-varying and time-invariant systems. Our results generalize the past works that consider the case with full or partial observation only, and the case with full observation and side information. Numerical simulations are presented to illustrate the effect of side information for partially observable systems.

cs.IT↗

arXiv2026

Attention-Aided MMSE with Ridge Denoising: How to Train under Noisy Channel Samples

Deep neural channel estimators are typically trained with clean channel state information (CSI), which is unavailable in practical orthogonal frequency-division multiplexing (OFDM) systems. In pilot-based OFDM, naive noisy-target training is structurally biased because the pilot input and noisy full-grid target share the same noise realization, driving the estimator toward identity copying. To address this for Attention-aided MMSE (A-MMSE), we propose a ridge-regularized objective that penalizes the generated filter directly. In a stylized fixed-filter model, this penalty induces scalar shrinkage and recovers the scalar MMSE gain at an explicit penalty value. We further construct surrogate training targets by estimating the channel covariance via eigenvalue clipping of the noisy empirical second-moment matrix, without requiring clean CSI labels. On COST 2100 channels, the proposed Ridge-A-MMSE consistently outperforms the Noise2Noise (N2N) baselines considered in this paper, and the combination of ridge regularization and covariance shrinkage approaches the same network trained with clean CSI labels at high signal-to-noise ratios (SNRs).

eess.SP↗

arXiv2026

Lower Bound of Networked Control with Multiple Sensors and One Controller And The Application to Tracking Gaussian-Markov Source

This paper investigates the causal rate-distortion function for networked control systems with multiple encoders and a single decoder, a longstanding open problem in information and control theory. While previous work has explored the causal rate-distortion function for single-encoder and feedback-enabled networked settings, the case of networks without feedback remains unaddressed. We establish a novel directed information lower bound, the first derived for the networked control setting. We further demonstrate the optimality of linear, independent encoders and linear decoders for optimizing this lower bound for Linear Quadratic Gaussian (LQG) plant and quadratic cost, with the condition that the full plant state is observed when sensors are sitting together. By reducing the original infinite-dimensional optimization problem to a finite-dimensional one, our approach simplifies the analysis. Additionally, our directed information lower bound provides an alternate proof for the sufficiency of linear encoders in the single encoder and single decoder setting with side information, extending prior results in the literature. We present Semidefinite Programming formulations for the causal rate distortion function of Gaussian-Markov sources with linear side information and the singular noise matrix.

cs.IT↗

arXiv2026

Efficient Weighted Sampling via Score-based Generative Models

Weighted sampling -- sampling from a probability density function (PDF) proportional to the product of a base PDF and a weight function -- is a fundamental technique with wide-ranging applications in variance reduction, biased sampling, data augmentation, and more. Leveraging the increasing availability of pretrained score-based generative models (SGMs), we propose a training-free weighted sampling framework that approximates the backward diffusion process of the target distribution by augmenting the pretrained base score function with an auxiliary guidance term, in a principled and computationally efficient manner. Our approach builds on two key components: a lightweight approximation of the guidance that avoids costly higher-order derivatives of both the score and weight functions, and an uncertainty-aware scheduler that dynamically adjusts the guidance strength based on a temporal analysis of approximation error. Together, these components enable accurate and stable sampling without relying on particle-based resampling or Hessian evaluations commonly required by existing methods. We validate the effectiveness of our method from synthetic to large-scale settings such as Stable Diffusion XL, where our framework achieves $1.2\times$ to $4.7\times$ speedups while consistently matching or outperforming state-of-the-art baselines in task performance. These results position our method as a scalable and inference-efficient solution for task-adaptive, time-sensitive sampling in generative applications.

cs.LG↗

arXiv2026

Fine-Tuning Masked Diffusion for Provable Self-Correction

A natural desideratum for generative models is self-correction--detecting and revising low-quality tokens at inference. While Masked Diffusion Models (MDMs) have emerged as a promising approach for generative modeling in discrete spaces, their capacity for self-correction remains poorly understood. Prior attempts to incorporate self-correction into MDMs either require overhauling MDM architectures/training or rely on imprecise proxies for token quality, limiting their applicability. Motivated by this, we introduce PRISM--Plug-in Remasking for Inference-time Self-correction of Masked Diffusions--a lightweight, model-agnostic approach that applies to any pretrained MDM. Theoretically, PRISM defines a self-correction loss that provably learns per-token quality scores, without RL or a verifier. These quality scores are computed in the same forward pass with MDM and used to detect low-quality tokens. Empirically, PRISM advances MDM inference across domains and scales: Sudoku; unconditional text (170M); and code with LLaDA (8B).

cs.LG↗

arXiv2026

TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning

Large Reasoning Models (LRMs) achieve impressive performance on complex reasoning tasks via Chain-of-Thought (CoT) reasoning, which enables them to generate intermediate thinking tokens before arriving at the final answer. However, LRMs often suffer from significant overthinking, spending excessive compute time even after the answer is generated early on. Prior work has identified the existence of an optimal reasoning length such that truncating reasoning at this point significantly shortens CoT outputs with virtually no change in performance. However, determining optimal CoT lengths for practical datasets is highly non-trivial as they are fully task and model-dependent. In this paper, we precisely address this and design Terminator, an early-exit strategy for LRMs at inference to mitigate overthinking. The central idea underpinning Terminator is that the first arrival of an LRM's final answer is often predictable, and we leverage these first answer positions to create a novel dataset of optimal reasoning lengths to train Terminator. Powered by this approach, Terminator achieves significant reductions in CoT lengths of 14%-55% on average across four challenging practical datasets: MATH-500, AIME 2025, HumanEval, and GPQA, while outperforming current state-of-the-art methods and reducing inference latency by more than 2x compared to the original LRM.

cs.LG↗

arXiv2026

Deep-OFDM: Neural Modulation for High Mobility

Orthogonal Frequency Division Multiplexing (OFDM) is the dominant waveform in modern wireless systems, but suffers performance degradation in high-mobility environments due to Doppler-induced inter-carrier interference and unreliable pilot-based channel estimation. Neural receivers have recently shown strong performance in OFDM systems by learning equalization and detection directly from the received time-frequency grid. However, when channel estimation becomes unreliable, receiver-side learning alone is insufficient to fully recover performance. In this work we introduce DeepOFDM, a learnable modulation framework that augments conventional OFDM with a lightweight convolutional neural network (CNN) modulator jointly optimized with a neural receiver. Instead of mapping symbols independently to resource elements, DeepOFDM spreads information across local time-frequency neighborhoods while remaining fully compatible with FFT-based OFDM processing. The learned modulation breaks the rotational symmetry of conventional QAM constellations, enabling the receiver to infer residual phase directly from data symbols. This structure allows reliable operation with sparse pilots and even in fully pilotless settings. Extensive simulations demonstrate improvements in block error rate and goodput under high Doppler, while over-the-air experiments confirm practical feasibility. These results highlight the potential of transmitter-receiver co-design for robust and spectrally efficient AI-native physical layer design.

cs.IT↗

arXiv2026

Dynamic Tokenization via Reinforcement Patching: End-to-end Training and Zero-shot Transfer

Efficiently aggregating spatial or temporal horizons to acquire compact representations has become a unifying principle in modern deep learning models, yet learning data-adaptive representations for long-horizon sequence data, especially continuous sequences like time series, remains an open challenge. While fixed-size patching has improved scalability and performance, discovering variable-sized, data-driven patches end-to-end often forces models to rely on soft discretization, specific backbones, or heuristic rules. In this work, we propose Reinforcement Patching (ReinPatch), the first framework to jointly optimize a sequence patching policy and its downstream sequence backbone model using reinforcement learning. By formulating patch boundary placement as a discrete decision process optimized via Group Relative Policy Gradient (GRPG), ReinPatch bypasses the need for continuous relaxations and performs dynamic patching policy optimization in a natural manner. Moreover, our method allows strict enforcement of a desired compression rate, freeing the downstream backbone to scale efficiently, and naturally supports multi-level hierarchical modeling. We evaluate ReinPatch on time-series forecasting datasets, where it demonstrates compelling performance compared to state-of-the-art data-driven patching strategies. Furthermore, our detached design allows the patching module to be extracted as a standalone foundation patcher, providing the community with visual and empirical insights into the segmentation behaviors preferred by a purely performance-driven neural patching strategy.

cs.LG↗

arXiv2026

Residual Diffusion Models for Variable-Rate Joint Source Channel Coding of MIMO CSI

Despite significant advancements in deep learning based CSI compression, some key limitations remain unaddressed. Current approaches predominantly treat CSI compression as a source-coding problem, thereby neglecting transmission errors. Conventional separate source and channel coding suffers from the cliff effect, leading to significant deterioration in reconstruction performance under challenging channel conditions. While existing autoencoder-based compression schemes can be readily extended to support joint source-channel coding, they struggle to capture complex channel distributions and exhibit poor scalability with increasing parameter count. To overcome these inherent limitations of autoencoder-based approaches, we propose Residual-Diffusion Joint Source-Channel Coding (RD- JSCC), a novel framework that integrates a lightweight autoencoder with a residual diffusion module to iteratively refine CSI reconstruction. Our flexible decoding strategy balances computational efficiency and performance by dynamically switching between low-complexity autoencoder decoding and sophisticated diffusion-based refinement based on channel conditions. Comprehensive simulations demonstrate that RD-JSCC significantly outperforms existing autoencoder-based approaches in challenging wireless environments. Furthermore, RD-JSCC offers several practical features, including a low-latency 2-step diffusion during inference, support for multiple compression rates with a single model, robustness to fixed-bit quantization, and adaptability to imperfect channel estimation.

cs.IT↗

arXiv2025

Generating High Dimensional User-Specific Wireless Channels using Diffusion Models

Deep neural network (DNN)-based algorithms are emerging as an important tool for many physical and MAC layer functions in future wireless communication systems, including for large multi-antenna channels. However, training such models typically requires a large dataset of high-dimensional channel measurements, which are very difficult and expensive to obtain. This paper introduces a novel method for generating synthetic wireless channel data using diffusion-based models to produce user-specific channels that accurately reflect real-world wireless environments. Our approach employs a conditional denoising diffusion implicit model (cDDIM) framework, effectively capturing the relationship between user location and multi-antenna channel characteristics. We generate synthetic high fidelity channel samples using user positions as conditional inputs, creating larger augmented datasets to overcome measurement scarcity. The utility of this method is demonstrated through its efficacy in training various downstream tasks such as channel compression and beam alignment. Our diffusion-based augmentation approach achieves over a 1-2 dB gain in NMSE for channel compression, and an 11dB SNR boost in beamforming compared to prior methods, such as noise addition or the use of generative adversarial networks (GANs).

cs.IT↗

arXiv2025

Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chains

Attention-based transformers have achieved tremendous success across a variety of disciplines including natural languages. To deepen our understanding of their sequential modeling capabilities, there is a growing interest in using Markov input processes to study them. A key finding is that when trained on first-order Markov chains, transformers with two or more layers consistently develop an induction head mechanism to estimate the in-context bigram conditional distribution. In contrast, single-layer transformers, unable to form an induction head, directly learn the Markov kernel but often face a surprising challenge: they become trapped in local minima representing the unigram distribution, whereas deeper models reliably converge to the ground-truth bigram. While single-layer transformers can theoretically model first-order Markov chains, their empirical failure to learn this simple kernel in practice remains a curious phenomenon. To explain this contrasting behavior of single-layer models, in this paper we introduce a new framework for a principled analysis of transformers via Markov chains. Leveraging our framework, we theoretically characterize the loss landscape of single-layer transformers and show the existence of global minima (bigram) and bad local minima (unigram) contingent on data properties and model architecture. We precisely delineate the regimes under which these local optima occur. Backed by experiments, we demonstrate that our theoretical findings are in congruence with the empirical results. Finally, we outline several open problems in this arena. Code is available at https://github.com/Bond1995/Markov .

cs.LG↗

arXiv2025

Generative Diffusion Model-based Compression of MIMO CSI

While neural lossy compression techniques have markedly advanced the efficiency of Channel State Information (CSI) compression and reconstruction for feedback in MIMO communications, efficient algorithms for more challenging and practical tasks-such as CSI compression for future channel prediction and reconstruction with relevant side information-remain underexplored, often resulting in suboptimal performance when existing methods are extended to these scenarios. To that end, we propose a novel framework for compression with side information, featuring an encoding process with fixed-rate compression using a trainable codebook for codeword quantization, and a decoding procedure modeled as a backward diffusion process conditioned on both the codeword and the side information. Experimental results show that our method significantly outperforms existing CSI compression algorithms, often yielding over twofold performance improvement by achieving comparable distortion at less than half the data rate of competing methods in certain scenarios. These findings underscore the potential of diffusion-based compression for practical deployment in communication systems.

cs.IT↗

arXiv2024

Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models

We formalize the problem of prompt compression for large language models (LLMs) and present a framework to unify token-level prompt compression methods which create hard prompts for black-box models. We derive the distortion-rate function for this setup as a linear program, and provide an efficient algorithm to compute this fundamental limit via the dual of the linear program. Using the distortion-rate function as the baseline, we study the performance of existing compression schemes on a synthetic dataset consisting of prompts generated from a Markov chain, natural language queries, and their respective answers. Our empirical analysis demonstrates the criticality of query-aware prompt compression, where the compressor has knowledge of the downstream task/query for the black-box LLM. We show that there is a large gap between the performance of current prompt compression methods and the optimal strategy, and propose Adaptive QuerySelect, a query-aware, variable-rate adaptation of a prior work to close the gap. We extend our experiments to a small natural language dataset to further confirm our findings on our synthetic dataset.

cs.LG↗

arXiv2024

Task-aware Distributed Source Coding under Dynamic Bandwidth

Efficient compression of correlated data is essential to minimize communication overload in multi-sensor networks. In such networks, each sensor independently compresses the data and transmits them to a central node due to limited communication bandwidth. A decoder at the central node decompresses and passes the data to a pre-trained machine learning-based task to generate the final output. Thus, it is important to compress the features that are relevant to the task. Additionally, the final performance depends heavily on the total available bandwidth. In practice, it is common to encounter varying availability in bandwidth, and higher bandwidth results in better performance of the task. We design a novel distributed compression framework composed of independent encoders and a joint decoder, which we call neural distributed principal component analysis (NDPCA). NDPCA flexibly compresses data from multiple sources to any available bandwidth with a single model, reducing computing and storage overhead. NDPCA achieves this by learning low-rank task representations and efficiently distributing bandwidth among sensors, thus providing a graceful trade-off between performance and bandwidth. Experiments show that NDPCA improves the success rate of multi-view robotic arm manipulation by 9% and the accuracy of object detection tasks on satellite imagery by 14% compared to an autoencoder with uniform bandwidth allocation.

cs.IT↗