What are the theoretical foundations behind DDIM?

DDIM (Denoising Diffusion Implicit Models) builds on diffusion-based generative models but introduces key theoretical changes to improve sampling efficiency. Traditional diffusion models like DDPM (Denoising Diffusion Probabilistic Models) simulate a Markov chain that gradually adds noise to data and then learns to reverse this process. However, generating samples with DDPM requires hundreds or thousands of steps, which is computationally expensive. DDIM addresses this by redefining the diffusion process as a non-Markovian chain, allowing fewer steps without sacrificing sample quality. This is achieved by decoupling the training objective from the sampling process, enabling deterministic or partially deterministic generation paths that skip intermediate steps.

The core innovation in DDIM lies in its non-Markovian forward process and deterministic reverse process. Unlike DDPM, where each step depends only on the previous state (a Markov assumption), DDIM allows steps to depend on multiple prior states. This flexibility is derived from reparameterizing the diffusion process using a differential equation framework. For example, DDIM treats the reverse process as solving an ordinary differential equation (ODE) that maps noise back to data. By fixing the noise schedule and variance parameters (instead of learning them), DDIM simplifies the sampling trajectory. This deterministic approach ensures that the same latent variables produce identical outputs, enabling features like consistent interpolation in latent space. For instance, developers can smoothly transition between two images by interpolating their latent codes, which is harder with stochastic DDPM sampling.

From a practical standpoint, DDIM retains the same training procedure as DDPM, meaning existing diffusion models can be adapted to use DDIM sampling without retraining. This compatibility is crucial for developers: a model trained with DDPM’s noise-prediction objective can generate samples faster using DDIM’s ODE-based sampling. For example, generating a high-quality image might take 50 steps with DDIM instead of 1,000 steps with DDPM. The tradeoff is a slight loss in sample diversity due to the deterministic nature, but the quality remains comparable. DDIM’s theoretical framework also introduces the concept of “sampling trajectories,” where developers can balance speed and quality by adjusting the number of steps or the noise schedule. This makes DDIM a versatile tool for applications requiring rapid iteration, such as real-time image editing or design prototyping.