How is noise incorporated into the diffusion process?

Noise is incorporated into the diffusion process through a series of incremental steps that gradually transform structured data (like images) into random noise. This is done in a forward process where noise is added systematically, and a learned reverse process removes it to reconstruct the original data. The core idea is to train a model to reverse the noising steps, enabling it to generate new data by starting from pure noise and iteratively refining it.

The forward process follows a predefined noise schedule, which determines how much Gaussian noise is added at each timestep. For example, at each step ( t ), the current data ( x_t ) is a weighted combination of the previous data ( x_{t-1} ) and a noise sample ( \epsilon ). Mathematically, this is often expressed as ( x_t = \sqrt{1 - \beta_t} \cdot x_{t-1} + \sqrt{\beta_t} \cdot \epsilon ), where ( \beta_t ) controls the noise strength at step ( t ). The ( \beta_t ) values are typically small and increase gradually over time, ensuring that the data transitions smoothly from structure to noise. Developers often use predefined schedules for ( \beta_t ), such as linear or cosine-based increases, to balance the rate of corruption across steps.

During training, the model learns to reverse this process by predicting the noise component ( \epsilon ) added at each step. For instance, given a noisy input ( x_t ) and timestep ( t ), a neural network (e.g., a U-Net) is trained to estimate ( \epsilon ). The loss function compares the predicted noise to the actual noise used in the forward process. Once trained, the model can generate data by starting with random noise ( x_T ) and iteratively applying the reverse process: at each step, it predicts the noise in ( x_t ), subtracts it, and refines ( x_{t-1} ). To introduce stochasticity during sampling, some methods add a small amount of new noise at each denoising step, ensuring diverse outputs. This combination of controlled noising and learned denoising enables diffusion models to generate high-quality, varied results.