# How to Implement Prompt Weighting and Negative Prompts in Stable Diffusion

> Discover how to implement prompt weighting and negative prompts in Stable Diffusion to gain precise control over your image generations. Learn to blend conditioning tensors for superior results.

- Repository: [CompVis - Computer Vision and Learning LMU Munich/stable-diffusion](https://github.com/CompVis/stable-diffusion)
- Tags: how-to-guide
- Published: 2026-03-02

---

**To implement prompt weighting and negative prompts in Stable Diffusion, encode individual text fragments separately using `model.get_learned_conditioning`, apply linear scaling weights to the resulting conditioning vectors, and combine them via classifier-free guidance by strategically blending positive, negative, and unconditional conditioning tensors before passing them to the sampler.**

The CompVis/stable-diffusion repository generates images by converting text prompts into learned conditioning vectors through the CLIP text encoder. By manipulating these vectors directly, you can implement prompt weighting and negative prompts for improved generation control without altering the underlying diffusion model architecture.

## Understanding the Conditioning Mechanism

Stable Diffusion builds its text-to-image conditioning from a **learned conditioning vector** produced by the CLIP text encoder. The core method responsible for this transformation lives in [`ldm/models/diffusion/ddpm.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/diffusion/ddpm.py) within the `get_learned_conditioning` function (lines 51-63). This method converts text strings into tensors that the diffusion model uses to guide image generation.

The model expects a single conditioning tensor `c` during the sampling process. Because the conditioning exists in CLIP latent space—a continuous vector space where linear combinations remain semantically meaningful—you can mathematically blend multiple encoded prompts to achieve granular control over the final output.

## Implementing Prompt Weighting

Prompt weighting allows you to assign different importance levels to various concepts within a single generation. Since the conditioning is purely linear, you can decompose a complex prompt into weighted components.

### Splitting and Encoding Sub-Prompts

First, split your prompt into semantic fragments. For example, instead of passing "a cat in a garden, painting style" as one string, separate it into weighted components:

1. "a cat" (weight 1.0)
2. "in a garden" (weight 0.8)
3. "painting style" (weight 1.2)

Encode each fragment individually by calling `get_learned_conditioning`:

```python
c1 = model.get_learned_conditioning(["a cat"])
c2 = model.get_learned_conditioning(["in a garden"])
c3 = model.get_learned_conditioning(["painting style"])

```

### Linear Combination of Weighted Vectors

Multiply each conditioning tensor by its scalar weight and sum the results to create the final positive conditioning:

```python

# Apply weights and combine

c_pos = (1.0 * c1) + (0.8 * c2) + (1.2 * c3)

```

This weighted sum `c_pos` replaces the standard single-prompt conditioning. The diffusion sampler treats this linear combination as a valid point in CLIP space, effectively emphasizing or de-emphasizing specific visual concepts during generation.

## Implementing Negative Prompts via Classifier-Free Guidance

Stable Diffusion supports **classifier-free guidance** (CFG), the mechanism that steers the model away from an unconditional (empty) baseline. The repository implements this in [`scripts/txt2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/txt2img.py), where the unconditional conditioning `uc` is created using an empty string when guidance is enabled:

```python

# From scripts/txt2img.py

if opt.scale != 1.0:
    uc = model.get_learned_conditioning(batch_size * [""])

```

### The Classifier-Free Guidance Mechanism

The `unconditional_guidance_scale` (commonly called `cfg_scale` or `opt.scale`) controls how strongly the model follows the conditional prompt versus the unconditional baseline. The sampler applies this guidance using the formula:

```

final_cond = (1 + scale) * c_pos - scale * uc

```

### Combining Negative and Unconditional Conditioning

To implement true negative prompts—actively suppressing specific concepts rather than simply reverting to empty noise—you encode the negative text and mathematically combine it with the unconditional vector:

```python

# Encode negative prompts with their own weights

c_neg_1 = model.get_learned_conditioning(["blurry"])
c_neg_2 = model.get_learned_conditioning(["low quality"])

# Weight and sum negative conditionings

c_neg = (1.5 * c_neg_1) + (1.0 * c_neg_2)

# Blend with unconditional (empty) conditioning

uc_combined = c_neg + uc

```

When passed to the sampler as the `unconditional_conditioning` parameter, the guidance formula effectively becomes:

```

final_cond = (1 + scale) * c_pos - scale * (c_neg + uc)

```

Setting `scale` > 1 amplifies the difference between your positive weighted conditioning and the negative-plus-unconditional baseline, yielding stronger suppression of unwanted concepts.

## Practical Implementation Code

The actual sampling call that consumes both conditionings resides in the inference scripts. Replace the standard `uc` with your weighted negative combination as follows:

```python

# Generate weighted positive conditioning

c_pos = sum(w * model.get_learned_conditioning([text]) 
            for w, text in zip(pos_weights, pos_prompts))

# Generate weighted negative + unconditional

uc_base = model.get_learned_conditioning(batch_size * [""])
c_neg = sum(w * model.get_learned_conditioning([text]) 
            for w, text in zip(neg_weights, neg_prompts))
uc_final = uc_base + c_neg

# Pass to sampler (DDIM example from ldm/models/diffusion/ddim.py)

samples_ddim, _ = sampler.sample(
    S=opt.ddim_steps,
    conditioning=c_pos,
    batch_size=opt.n_samples,
    shape=shape,
    unconditional_guidance_scale=opt.scale,
    unconditional_conditioning=uc_final,
    eta=opt.ddim_eta,
    x_T=start_code
)

```

This approach works with any sampler implementation that honors the `unconditional_guidance_scale` and `unconditional_conditioning` parameters, including those in [`ldm/models/diffusion/ddim.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/diffusion/ddim.py) and [`ldm/models/diffusion/plms.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/diffusion/plms.py).

## Spatial Prompt Weighting Considerations

The repository also includes a spatial weighting scheme designed for large images processed in patches. While distinct from textual prompt weighting, this mechanism can complement your implementation. The relevant parameters are defined in [`notebook_helpers.py`](https://github.com/CompVis/stable-diffusion/blob/main/notebook_helpers.py) under `model.split_input_params` (including `clip_max_weight` and `clip_min_weight`), with the actual weighting applied inside `get_weighting` in [`ddpm.py`](https://github.com/CompVis/stable-diffusion/blob/main/ddpm.py). You can combine spatial weights with the textual conditioning weights described above to emphasize specific image regions.

## Summary

- **Prompt weighting** works by encoding sub-prompts separately via `get_learned_conditioning` in [`ldm/models/diffusion/ddpm.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/diffusion/ddpm.py), then computing a linear weighted sum of the resulting vectors.
- **Negative prompts** leverage the existing classifier-free guidance mechanism by replacing the standard empty unconditional conditioning with a weighted blend of negative concepts and the empty prompt baseline.
- The `unconditional_guidance_scale` parameter controls the strength of guidance, mathematically implementing the formula `(1 + scale) * c_pos - scale * uc`.
- No modifications to the core diffusion U-Net or sampling logic are required; you only manipulate the conditioning tensors before passing them to the sampler in [`scripts/txt2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/txt2img.py) or custom inference code.

## Frequently Asked Questions

### How does prompt weighting work mathematically in Stable Diffusion?

Prompt weighting exploits the linearity of the CLIP text encoder's output space. Because the conditioning vectors exist in a continuous latent space, the model accepts linear combinations of encoded prompts as valid inputs. You calculate `c = sum(w_i * encode(prompt_i))`, where `encode()` calls `get_learned_conditioning`. The diffusion model processes this weighted tensor exactly like a standard single-prompt conditioning vector.

### What is the difference between negative prompts and unconditional conditioning?

**Unconditional conditioning** traditionally uses an empty string to represent the model's baseline "no prompt" state, allowing classifier-free guidance to steer generation toward the positive prompt. **Negative prompts** actively encode text you want to suppress (such as "blurry" or "low quality") and mathematically add these to the unconditional vector. This creates `uc_final = uc_empty + c_neg`, causing the guidance mechanism to push the generation away from both random noise and specific unwanted concepts.

### Do I need to modify the core diffusion model to use prompt weights?

No. The CompVis/stable-diffusion implementation treats conditioning as an input tensor that you can manipulate externally. By modifying the tensors passed to the sampler's `conditioning` and `unconditional_conditioning` arguments in [`scripts/txt2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/txt2img.py) or custom wrappers, you implement weighting without changing [`ddpm.py`](https://github.com/CompVis/stable-diffusion/blob/main/ddpm.py), [`ddim.py`](https://github.com/CompVis/stable-diffusion/blob/main/ddim.py), or the U-Net architecture.

### Where is the classifier-free guidance scale applied in the code?

The `unconditional_guidance_scale` is applied inside the sampler implementations, specifically within [`ldm/models/diffusion/ddim.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/diffusion/ddim.py) and [`ldm/models/diffusion/plms.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/diffusion/plms.py). These samplers receive both `conditioning` (positive) and `unconditional_conditioning` (negative/unconditional) tensors, then apply the guidance formula during each denoising step to compute the conditioned score estimate.