Structured drift design for denoising diffusion models

Abstract

Diffusion-based generative models have achieved remarkable success in high-dimensional data generation; however, they fundamentally rely on isotropic diffusion processes that destroy meaningful geometric structures in the forward process. For complex, multimodal, and highly correlated distributions such as biologically constrained genetic data, isotropic noise merges distinct modes and distorts intrinsic dependencies. This forces the reverse process to recover structure from heavily degraded signals, leading to slow convergence, mode averaging, and biologically implausible samples. To address this, we introduce the Geometry-aware Ornstein-Uhlenbeck (GOU) process, a structured drift design that embeds data geometry into forward and backward dynamics. By employing a variance-aware anisotropic drift, GOU contracts low-variance directions rapidly while preserving high-variance directions longer, maintaining key multimodal structures as stable channels over time. Crucially, we show that GOU's backward initialization error is governed by local rather than global variance. This geometry-adaptive initialization improves convergence rates by reducing initial mismatch and preserving cluster-level structures. Synthetic and real-world genetic experiments demonstrate that GOU significantly improves mode separation, correlation preservation, and statistical validity over standard isotropic models.