How do deterministic sampling methods (like DDIM) differ from stochastic ones?

Deterministic sampling methods, such as DDIM (Denoising Diffusion Implicit Models), differ from stochastic approaches primarily in how they handle randomness during the generation process. Deterministic methods produce the same output every time for a given starting point (e.g., a noise vector), because they eliminate random sampling steps. Stochastic methods, like those used in DDPM (Denoising Diffusion Probabilistic Models), introduce randomness at each step of the generation process, leading to varied outputs even with the same initial conditions. This distinction fundamentally changes how these models balance speed, consistency, and sample diversity.

A key technical difference lies in the sampling process. DDIM, for example, redefines the diffusion process to use a non-Markovian chain, allowing it to skip intermediate steps without relying on random noise additions. This makes the sampling path deterministic: once the initial noise and model parameters are fixed, the output is reproducible. In contrast, stochastic methods like DDPM follow a Markov chain, where each step depends on random noise injected during the forward process. For instance, DDPM might add Gaussian noise at every timestep, and reversing this requires sampling from a distribution that accounts for that randomness. This makes stochastic methods inherently slower but capable of producing more varied outputs.

The trade-offs between these approaches depend on use cases. Deterministic sampling (e.g., DDIM) is faster and more efficient because it can generate samples in fewer steps (e.g., 50 steps instead of 1000) while maintaining quality. This is useful for applications like real-time image editing or scenarios where reproducibility matters. Stochastic methods, while slower, excel in generating diverse samples, which is critical for tasks like creative art generation. For example, a developer might choose DDIM for a video synthesis tool requiring frame consistency, but opt for a stochastic method when generating multiple unique design variants.