How does classifier-free guidance differ from classifier guidance?

Classifier-free guidance and classifier guidance are two techniques used to steer the output of generative models, like diffusion models, toward desired attributes (e.g., generating images of a specific class). The key difference lies in how they incorporate conditional information during generation. Classifier guidance relies on an external classifier to influence sampling, while classifier-free guidance builds conditioning directly into the model, eliminating the need for a separate classifier.

Classifier guidance works by combining the generative model’s output with gradients from a pre-trained classifier. For example, if generating an image of a “cat,” the classifier evaluates how “cat-like” intermediate noisy samples are during the denoising process. The model then adjusts its output using the classifier’s gradients to amplify the “cat” features. This approach requires training a classifier on noisy data (matching the diffusion process’s timesteps), which adds complexity. It can also be brittle: if the classifier isn’t robust to noise or misaligned with the generative model, guidance may fail. For instance, a ResNet classifier trained on CIFAR-10 might struggle to provide reliable gradients for high-noise samples, leading to artifacts.

Classifier-free guidance avoids these issues by training the generative model to handle both conditional and unconditional generation. During training, the model randomly ignores the class label (or text prompt) in some batches, forcing it to learn unconditional generation alongside class-specific outputs. At inference time, the model blends these modes: for example, a text-to-image model like Stable Diffusion uses a guidance scale to interpolate between its unconditional prediction (ignoring the prompt) and its conditional prediction (following the prompt). This eliminates the need for a separate classifier, simplifying deployment. However, it requires careful tuning of the guidance scale—too high a value can overemphasize the condition, producing oversaturated or unrealistic outputs.

In summary, classifier guidance uses external models to steer generation, introducing complexity but allowing post-training control. Classifier-free guidance integrates conditioning into the generative model itself, trading higher training costs for simpler, more stable inference. Developers might choose classifier-free guidance for tasks like text-to-image generation (where training data is abundant) but could opt for classifier guidance when fine-grained control via external models is preferable.