A unified discrete diffusion framework that outperforms autoregressive models on large-scale discrete generation tasks for the first time.
March 24, 2026
Original Paper
Generalized Discrete Diffusion from Snapshots
arXiv · 2603.21342
The Takeaway
Autoregressive models have dominated large-scale text and discrete data generation, while discrete diffusion has struggled to scale. This framework uses a 'snapshot' latent derivation to simplify training and increase efficiency, demonstrating that non-autoregressive discrete diffusion is now a viable, and potentially superior, alternative to standard AR models at scale.
From the abstract
We introduce Generalized Discrete Diffusion from Snapshots (GDDS), a unified framework for discrete diffusion modeling that supports arbitrary noising processes over large discrete state spaces. Our formulation encompasses all existing discrete diffusion approaches, while allowing significantly greater flexibility in the choice of corruption dynamics. The forward noising process relies on uniformization and enables fast arbitrary corruption. For the reverse process, we derive a simple evidence l