Classifier-Free Guidance in Stable Diffusion: How Guidance Scale Controls Generation Quality

Classifier-free guidance steers Stable Diffusion outputs toward a text prompt by linearly interpolating between unconditional and conditional noise predictions, where the guidance scale determines the trade-off between prompt fidelity and image diversity.

Classifier-free guidance (CFG) is a sampling technique implemented in the CompVis/stable-diffusion repository that enables text-to-image generation without requiring a separate classifier network. By combining conditional and unconditional predictions during inference, CFG allows users to control how strictly the generated image adheres to the input prompt through a single hyperparameter: the guidance scale.

How Classifier-Free Guidance Works

During training, the U-Net learns two distinct behaviors. In conditional mode, the model predicts noise ε(xₜ, t, c) given the latent xₜ, timestep t, and conditioning c (typically CLIP text embeddings). In unconditional mode, the model predicts ε(xₜ, t, ∅) where the conditioning is randomly dropped (typically 10% of training batches) and replaced with a null vector.

At inference, the final noise estimate combines both predictions:

$$ \hat\varepsilon = \varepsilon_{\text{uncond}} + s , (\varepsilon_{\text{cond}} - \varepsilon_{\text{uncond}}) $$

Here, s represents the guidance scale. When s = 1, the formula collapses to the pure conditional prediction. Values greater than 1 amplify the influence of the text prompt by pushing the prediction away from the unconditional direction.

Implementation in the Stable Diffusion Codebase

The CFG logic is implemented in the sampler wrappers within the CompVis/stable-diffusion repository.

In ldm/models/diffusion/dpm_solver/dpm_solver.py, lines 236-244, the model_wrapper class handles the conditional-free combination:

# From ldm/models/diffusion/dpm_solver/dpm_solver.py (lines 236-244)

if unconditional_conditioning is not None:
    # Unconditional prediction

    e_t_uncond = self.model(x, t, unconditional_conditioning)
    # Conditional prediction

    e_t_cond = self.model(x, t, cond)
    # Classifier-free guidance combination

    e_t = e_t_uncond + unconditional_guidance_scale * (e_t_cond - e_t_uncond)
else:
    e_t = self.model(x, t, cond)

Similarly, in ldm/models/diffusion/plms.py, lines 78-90, the PLMS sampler applies the same pattern:

# From ldm/models/diffusion/plms.py (lines 78-90)

if unconditional_guidance_scale != 1.0:
    # Concatenate conditional and unconditional for efficient batching

    x_in = torch.cat([x] * 2)
    t_in = torch.cat([t] * 2)
    c_in = torch.cat([unconditional_conditioning, cond])
    e_t_uncond, e_t = self.model(x_in, t_in, c_in).chunk(2)
    e_t = e_t_uncond + unconditional_guidance_scale * (e_t - e_t_uncond)

How the Guidance Scale Affects Generation Quality

The guidance scale acts as a dial controlling the trade-off between sample diversity and prompt adherence. The CompVis/stable-diffusion codebase defaults to s = 7.5, but understanding the full spectrum helps optimize outputs for specific use cases.

Guidance Scale	Effect on Output
1.0	Pure conditional sampling. Images exhibit high diversity but may drift from the prompt description. Useful for exploratory generation where strict adherence is not required.
1.0 – 5.0	Moderate guidance. Prompt fidelity improves significantly while maintaining reasonable diversity. Good balance for most artistic applications.
5.0 – 7.5	Strong guidance (default range). Fine-grained details match the prompt closely. At the upper end of this range, subtle artifacts like over-sharpening or color saturation may begin to appear.
> 7.5	Excessive guidance. The model may produce over-saturated colors, repetitive patterns, or anatomical distortions. The diffusion ODE becomes over-constrained, reducing the stochastic smoothing that produces natural textures.

When s > 7, the term (ε_cond - ε_uncond) dominates the noise prediction, effectively treating the unconditional path as a negative anchor. This gradient ascent toward the conditional likelihood can overshoot the natural data manifold, creating the characteristic "CFG artifacts" seen in high-guidance generations.

Practical Code Examples

Command-Line Interface with txt2img.py

The simplest way to adjust guidance scale is via the --scale argument in the provided inference script:

python scripts/txt2img.py \
  --prompt "a cyberpunk cityscape at night, neon lights, highly detailed" \
  --ckpt models/ldm/stable-diffusion-v1/model.ckpt \
  --scale 7.5 \
  --n_samples 4 \
  --n_iter 1

In scripts/txt2img.py (lines 96-108), this parameter initializes the unconditional conditioning vector and passes it to the sampler:

# From scripts/txt2img.py (lines 96-108)

uc = model.get_learned_conditioning(batch_size * [""])  # Empty string = unconditional

if scale != 1.0:
    samples_ddim, _ = sampler.sample(
        S=ddim_steps,
        conditioning=c,
        batch_size=n_samples,
        shape=shape,
        verbose=False,
        unconditional_guidance_scale=scale,
        unconditional_conditioning=uc,
        eta=ddim_eta
    )

Python API with PLMS Sampler

For programmatic control, instantiate the PLMSSampler and explicitly set unconditional_guidance_scale:

import torch
from ldm.models.diffusion.plms import PLMSSampler
from ldm.util import instantiate_from_config
from omegaconf import OmegaConf

# Load model configuration and checkpoint (example setup)

config = OmegaConf.load("configs/stable-diffusion/v1-inference.yaml")
model = instantiate_from_config(config.model)
model.load_state_dict(torch.load("models/ldm/stable-diffusion-v1/model.ckpt")["state_dict"])
model = model.cuda().eval()

# Prepare conditioning tensors

prompt = "a majestic dragon breathing fire, digital art"
cond = model.get_learned_conditioning([prompt])
uncond = model.get_learned_conditioning([""])  # Null conditioning

# Initialize sampler with high guidance

sampler = PLMSSampler(model)
samples, intermediates = sampler.sample(
    S=50,                           # Sampling steps

    conditioning=cond,
    batch_size=1,
    shape=[4, 64, 64],             # Latent shape (C, H, W)

    unconditional_guidance_scale=9.0,  # Strong guidance (>7.5)

    unconditional_conditioning=uncond,
    verbose=False
)

# Decode to pixel space

x_samples = model.decode_first_stage(samples)
images = torch.clamp((x_samples + 1.0) / 2.0, min=0.0, max=1.0)

The critical logic resides in ldm/models/diffusion/plms.py lines 78-90, where the sampler concatenates conditional and unconditional inputs for efficient batch processing, then applies the guidance scale to the noise prediction difference.

Low-Level Model Wrapper for Custom Samplers

When implementing custom sampling loops or integrating with external ODE solvers, use the model_wrapper from the DPM-Solver implementation:

from ldm.models.diffusion.dpm_solver.dpm_solver import model_wrapper, DPM_Solver

# Assuming model, noise_schedule, cond, and uncond are defined

wrapped_model = model_wrapper(
    model=model,
    noise_schedule="discrete",
    model_type="noise",
    guidance_type="classifier-free",
    condition=cond,
    unconditional_condition=uncond,
    guidance_scale=12.0,   # Very strong guidance

)

# Initialize DPM-Solver with the wrapped model

dpm_solver = DPM_Solver(model_fn=wrapped_model, ...)
x_T = torch.randn(batch_size, 4, 64, 64).cuda()

# Run sampling

x_0 = dpm_solver.sample(x_T, steps=20, ...)

This wrapper implements the exact combination formula from lines 236-244 of dpm_solver.py, handling the conditional and unconditional forward passes internally.

Summary

• Classifier-free guidance eliminates the need for external classifiers by training the diffusion model on both conditional and unconditional objectives, randomly dropping conditioning during training (typically 10% of the time).
• The guidance scale (s) controls the interpolation between unconditional and conditional noise predictions via the formula $\hat\varepsilon = \varepsilon_{\text{uncond}} + s(\varepsilon_{\text{cond}} - \varepsilon_{\text{uncond}})$.
• In the CompVis/stable-diffusion codebase, this logic is implemented in ldm/models/diffusion/plms.py (lines 78-90) and ldm/models/diffusion/dpm_solver/dpm_solver.py (lines 236-244).
• Low scales (1-5) prioritize diversity and artistic freedom, while high scales (7.5-12) enforce strict prompt adherence at the risk of saturation, sharpness artifacts, or anatomical distortions.
• The default value of 7.5 provides a balanced starting point for most generation tasks.

Frequently Asked Questions

What is the difference between classifier guidance and classifier-free guidance?

Traditional classifier guidance requires a separately trained classifier to compute gradients during sampling, adding complexity and computational overhead. Classifier-free guidance, as implemented in CompVis/stable-diffusion, eliminates this external dependency by training the U-Net jointly on conditional and unconditional objectives. During inference, the model uses the difference between its own conditional and unconditional predictions to approximate the classifier gradient, simplifying the architecture while maintaining controllability.

Why does a high guidance scale cause image artifacts?

When the guidance scale exceeds approximately 7.5, the amplification of the conditional signal overshoots the natural data manifold, effectively over-constraining the diffusion ODE. This excessive gradient ascent toward the prompt likelihood reduces the stochastic smoothing that creates natural textures. The result is over-saturated colors, excessive sharpness, repetitive patterns, and anatomical distortions that characterize "CFG artifacts."

What is the default guidance scale in Stable Diffusion?

The CompVis/stable-diffusion reference implementation and most derived interfaces (including the original txt2img.py script) default to a guidance scale of 7.5. This value represents an empirical balance between prompt fidelity and image quality for the original LAION-5B trained checkpoints. Users can adjust this via the --scale command-line argument or the unconditional_guidance_scale parameter in the Python API.

Can I use classifier-free guidance with any diffusion sampler?

While the mathematical framework of classifier-free guidance is sampler-agnostic, practical implementation requires the sampler to support the conditional-unconditional combination logic. The CompVis/stable-diffusion codebase provides native CFG support in the PLMS and DPM-Solver samplers through the unconditional_guidance_scale parameter. Custom samplers must explicitly implement the noise prediction combination formula to utilize this feature, though the underlying U-Net architecture remains compatible regardless of the sampling algorithm chosen.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}