arXiv · 2610.19020
Discrete Diffusion Models are Minimax Optimal Under Sparsity
Abstract
Discrete diffusion models have emerged as a leading alternative to autoregressive models for generating text and other categorical data, yet their theoretical foundations remain less developed than those of their continuous counterparts. For continuous diffusion models, a prominent line of theory suggests that these models can adapt to low-dimensional structure in the data, thereby improving statistical efficiency and generalization. In this work, we investigate the discrete analogue of this phenomenon by studying the statistical rates of discrete diffusion models under sparsity assumptions. By analyzing a nonparametric estimator derived from a regularized score-entropy objective, we establish minimax-optimal rates in the sparse regime. The proposed regularization can be viewed as an adaptation of label smoothing techniques introduced for classification, and does not require the knowledge of the sparsity parameter. This shows that discrete diffusion models can adapt intrinsically to sparsity.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Luca Blunda, Tyler Farghly, Umut Simsekli, Alain Durmus. 2026-10-13. Discrete Diffusion Models are Minimax Optimal Under Sparsity. https://arxiv.science/abs/2610.19020
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.