What is classifier guidance in diffusion models?

Classifier guidance is a technique used in diffusion models to steer the generation process toward specific outputs by incorporating information from a pre-trained classifier. Diffusion models generate data by iteratively refining random noise into structured outputs, such as images, through a series of denoising steps. Classifier guidance modifies this process by using gradients from a classifier—trained on the same data as the diffusion model—to adjust the denoising steps. This ensures the generated sample aligns with a desired class label or attribute, effectively “guiding” the model to produce outputs that match specific criteria.

The technical implementation involves calculating the gradient of the classifier’s predicted probability for a target class with respect to the intermediate noisy data at each denoising step. For example, if generating an image of a dog, the classifier’s gradient indicates how to tweak the current noisy image to increase the probability it’s classified as a dog. These gradients are scaled by a guidance strength parameter (often denoted as ( s )) and combined with the diffusion model’s own prediction for the denoising step. Mathematically, the adjusted denoising direction becomes the original model prediction plus ( s \times \nabla_x \log p_\text{class}(y \mid x_t) ), where ( x_t ) is the noisy data at step ( t ) and ( y ) is the target class. This hybrid update nudges the generation process toward regions of the data space that satisfy both the diffusion model’s likelihood and the classifier’s criteria.

A practical example is using classifier guidance with a diffusion model trained on a dataset like CIFAR-10. Suppose you want to generate images of trucks. Without guidance, the model samples randomly across all classes. By adding a classifier trained to distinguish CIFAR-10 classes, you can compute gradients that push each denoising step toward features typical of trucks (e.g., wheels, flatbeds). The guidance strength ( s ) controls the trade-off: higher values produce images that more closely match the class but may reduce diversity or introduce artifacts if the classifier is imperfect. One limitation is the need for a separate classifier, which must be compatible with the diffusion model’s training data and architecture. Despite this, classifier guidance remains a flexible tool for controlled generation, especially when precise attribute targeting is required.