What is denoising diffusion probabilistic modeling (DDPM)?

Denoising Diffusion Probabilistic Models (DDPMs) are a class of generative models designed to generate data by learning to reverse a gradual noising process. Unlike models that generate data in a single step, DDPMs operate through a sequence of steps, progressively transforming random noise into structured data. The core idea involves two phases: a forward process that systematically adds noise to data until it becomes random, and a reverse process that trains a neural network to iteratively remove this noise. This approach allows the model to learn data distributions by focusing on small, incremental changes rather than attempting direct generation.

The forward process in DDPMs is defined as a Markov chain that gradually corrupts input data (e.g., an image) over multiple timesteps by adding Gaussian noise. For example, starting with a clean image, each step applies a small amount of noise, controlled by a predefined schedule, until the data becomes indistinguishable from random noise. The reverse process trains a neural network—often a U-Net—to predict the noise added at each step, enabling it to “denoise” samples. During training, the model minimizes the difference between the predicted noise and the actual noise introduced at each timestep, typically using a mean squared error loss. This stepwise approach makes the problem more tractable compared to single-step generation.

DDPMs have been applied to tasks like image synthesis, inpainting, and super-resolution. For instance, they can generate high-quality images by starting with random noise and applying the learned denoising steps. A key advantage is stability during training, as the model avoids adversarial setups like those in GANs. However, generating samples can be slow due to the iterative process—often requiring hundreds of steps. Recent improvements, such as DDIM (Denoising Diffusion Implicit Models), address this by enabling faster sampling with fewer steps while maintaining quality. Developers implementing DDPMs typically work with frameworks like PyTorch, leveraging existing libraries for noise scheduling and U-Net architectures to streamline training and inference.