The forward diffusion process is a mathematical framework that gradually transforms data into noise over a series of steps. It is defined as a Markov chain—a sequence of steps where each step depends only on the previous one—that incrementally adds Gaussian noise to an input dataset. Starting from an initial data point ( x_0 ), the process applies noise at each timestep ( t ) according to a predefined schedule, ultimately converting the data into a sample resembling pure noise after ( T ) steps. This transformation is deterministic and controlled by variance parameters, ensuring the data’s structure is systematically destroyed in a controlled manner.
Mathematically, the forward process is defined using a noise schedule ( \beta_t ), which determines how much noise is added at each step ( t ). For a given timestep ( t ), the noised data ( x_t ) is sampled from a Gaussian distribution conditioned on the previous step ( x_{t-1} ): [ q(x_t | x_{t-1}) = \mathcal{N}(x_t; \sqrt{1 - \beta_t} \cdot x_{t-1}, \beta_t \mathbf{I}) ] Here, ( \sqrt{1 - \beta_t} ) scales the previous data point to preserve its magnitude, while ( \beta_t ) controls the variance of the added noise. To simplify sampling at arbitrary timesteps, a cumulative product ( \alpha_t = \prod_{s=1}^t (1 - \beta_s) ) is used, allowing ( x_t ) to be expressed directly in terms of ( x_0 ): [ x_t = \sqrt{\alpha_t} \cdot x_0 + \sqrt{1 - \alpha_t} \cdot \epsilon ] where ( \epsilon \sim \mathcal{N}(0, \mathbf{I}) ). This formulation avoids iterating through all intermediate steps, making computations efficient. For example, a linear noise schedule might set ( \beta_t ) to increase from ( 10^{-4} ) to ( 0.02 ) over ( T=1000 ) steps.
In practice, the noise schedule ( \beta_t ) is critical to the process’s behavior. A poorly chosen schedule (e.g., too aggressive) can destroy data structure too quickly, making it harder for the reverse process to recover meaningful signals. Developers often experiment with schedules like linear, cosine, or learned adaptive schemes. The forward process itself requires no trainable parameters—it’s a fixed computation that enables training a neural network to reverse the noise addition. By the final step ( T ), ( \alpha_T ) approaches zero, and ( x_T ) approximates standard Gaussian noise ( \mathcal{N}(0, \mathbf{I}) ), providing a clear starting point for the reverse diffusion process. This setup is foundational in diffusion models, allowing them to generate data by learning to iteratively denoise samples.