# Internal Architecture of the Image Generation Processing Pipeline in AUTOMATIC1111’s Stable Diffusion WebUI

> Explore AUTOMATIC1111 Stable Diffusion WebUI's nine-stage internal architecture detailing text-to-image processing from front-end to post-generation scripts.

- Repository: [AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui)
- Tags: architecture
- Published: 2026-02-24

---

**The AUTOMATIC1111 Stable Diffusion WebUI executes text-to-image generation through a modular nine-stage pipeline that decouples the Gradio front-end, model hijacking layers, latent diffusion sampling, and extensible post-processing scripts.**

The **AUTOMATIC1111/stable-diffusion-webui** repository provides one of the most widely adopted open-source interfaces for Stable Diffusion. Understanding its internal architecture reveals how raw text prompts traverse through tokenization, model hijacking, latent diffusion, and post-processing to produce final images, with each stage isolated to allow surgical customization.

## The Nine-Stage Pipeline Architecture

The WebUI deliberately separates concerns into nine distinct stages, allowing developers to swap samplers, inject custom logic, or replace post-processors without modifying core diffusion code.

### Stage 1: Front-End Request Handling

When a user clicks **Generate**, the **Gradio** interface defined in [`webui.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/webui.py) captures all UI parameters—prompt, negative prompt, sampler name, CFG scale, and dimensions. This request is queued as a generation job and routed through [`launch.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/launch.py) to the appropriate backend handler. For API consumers, the REST endpoints defined in [`modules/api/api.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/api/api.py) expose the same functionality via `/sdapi/v1/txt2img` and `/sdapi/v1/img2img`.

- **Key files:** [`webui.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/webui.py), [`launch.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/launch.py), [`modules/api/api.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/api/api.py)

### Stage 2: Processing Object Creation

The backend instantiates a **`Processing`** object that acts as the single source of truth for all generation parameters. For text-to-image tasks, [`modules/txt2img.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/txt2img.py) creates a `StableDiffusionProcessingTxt2Img` instance, while image-to-image flows use [`modules/img2img.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/img2img.py). This object receives a unique task ID and stores the loaded model reference from `shared.sd_model`.

- **Key files:** [`modules/txt2img.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/txt2img.py), [`modules/img2img.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/img2img.py), [`modules/processing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/processing.py)
- **Core class:** `StableDiffusionProcessingTxt2Img`

### Stage 3: Model Loading and the SD-Hijack Layer

Before inference begins, the system retrieves the selected checkpoint via [`sd_models.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/sd_models.py). The critical **SD-Hijack** layer then intercepts the model's native `forward` methods. Located in [`modules/sd_hijack.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_hijack.py) with specialized implementations in [`modules/sd_hijack_unet.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_hijack_unet.py) and [`modules/sd_hijack_clip.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_hijack_clip.py), this layer swaps the default UNet, VAE, and CLIP forward passes with wrapped versions that support **classifier-free guidance (CFG)**, prompt weighting, and custom attention mechanisms.

- **Key files:** [`modules/sd_hijack.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_hijack.py), [`modules/sd_hijack_unet.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_hijack_unet.py), [`modules/sd_hijack_clip.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_hijack_clip.py)

### Stage 4: Prompt Tokenization and Conditioning

The textual prompt undergoes parsing in [`modules/prompt_parser.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/prompt_parser.py), which handles syntax features like attention weighting `(word:1.2)` and wildcards. The parsed tokens are then encoded by the hijacked CLIP model into **conditioning vectors** (`c_crossattn` and `c_concat`). These vectors guide the diffusion process toward the semantic content described by the user.

- **Key file:** [`modules/prompt_parser.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/prompt_parser.py)

### Stage 5: The Latent Diffusion Sampling Loop

The core generation occurs in the sampler loop implemented across [`modules/sd_samplers.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_samplers.py) and backend-specific files like [`modules/sd_samplers_kdiffusion.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_samplers_kdiffusion.py). The selected sampler (Euler, DDIM, DPM++, etc.) iteratively denoises a latent tensor. At each step, the sampler invokes the **hijacked UNet** `forward` method, passing the current latent, timestep, and conditioning vectors. The wrapper applies the CFG scale by running the model twice—once with conditioned and once with unconditioned prompts—and blends the predictions before updating the latent.

- **Key files:** [`modules/sd_samplers.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_samplers.py), [`modules/sd_samplers_kdiffusion.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_samplers_kdiffusion.py)

### Stage 6: VAE Decoding and Latent-to-Pixel Conversion

Once the sampling loop completes, the final latent tensor resides in a compressed representation. The **VAE decoder** defined in [`modules/sd_vae.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_vae.py) converts these latents into a full RGB image. This stage operates on the GPU and represents the transition from the model's internal representation to human-viewable pixels.

- **Key file:** [`modules/sd_vae.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_vae.py)

### Stage 7: Safety Checking and Invisible Watermarking

Before returning results, the image passes through the safety pipeline in [`modules/safe.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/safe.py). If enabled, the **safety checker** analyzes the image for NSFW content. Simultaneously, an invisible watermark is embedded for provenance tracking. These steps ensure compliance and traceability without altering the visual appearance of the generated image.

- **Key file:** [`modules/safe.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/safe.py)

### Stage 8: Post-Processing and Restoration Scripts

The WebUI supports extensible post-processing through the `ScriptPostprocessing` interface defined in [`modules/scripts_postprocessing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/scripts_postprocessing.py). Optional steps such as upscaling via Real-ESRGAN ([`scripts/postprocessing_upscale.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/scripts/postprocessing_upscale.py)) or face restoration using GFPGAN/CodeFormer ([`scripts/postprocessing_gfpgan.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/scripts/postprocessing_gfpgan.py)) execute sequentially on the decoded image. Each script implements standardized `process()` and `title()` methods, allowing the UI to discover and chain them automatically.

- **Key files:** [`modules/scripts_postprocessing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/scripts_postprocessing.py), [`scripts/postprocessing_upscale.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/scripts/postprocessing_upscale.py)

### Stage 9: Metadata Packaging and Result Delivery

In the final stage, [`modules/infotext_utils.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/infotext_utils.py) assembles generation metadata—including seed, CFG scale, sampler name, and the full prompt—into an **infotext** string. This metadata is embedded within the image file (PNG metadata chunks) and returned to the Gradio front-end or REST API consumer. The complete image, along with its generation parameters, is now available for download, display, or further API processing.

- **Key file:** [`modules/infotext_utils.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/infotext_utils.py)

## Data Flow and Component Interaction

Understanding the sequential data flow clarifies how these stages interact:

1. **User initiates generation** — The Gradio interface in [`webui.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/webui.py) creates a `txt2img` or `img2img` call, or the REST API in [`modules/api/api.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/api/api.py) receives a JSON payload.

2. **Parameter encapsulation** — The system builds a `Processing` object (from [`modules/processing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/processing.py)) containing all generation parameters and a unique task ID.

3. **Model retrieval** — The `shared.sd_model` is retrieved or swapped via [`sd_models.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/sd_models.py), preparing the UNet, VAE, and CLIP for inference.

4. **Hijack injection** — The SD-Hijack layer replaces the model's native `forward` methods with wrapped versions that accept conditioning vectors and CFG scales.

5. **Conditioning generation** — The prompt flows through [`modules/prompt_parser.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/prompt_parser.py) into the hijacked CLIP encoder, producing `c_crossattn` and `c_concat` conditioning vectors.

6. **Latent denoising** — The sampler loop (in [`modules/sd_samplers.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_samplers.py)) repeatedly calls the hijacked UNet to predict noise, applies CFG by comparing conditioned and unconditioned predictions, and updates the latent tensor.

7. **Decoding** — The final latent passes through the VAE decoder in [`modules/sd_vae.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_vae.py) to produce RGB pixels.

8. **Safety and post-processing** — The image flows through [`modules/safe.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/safe.py) for content filtering, then through the post-processing script chain defined in [`modules/scripts_postprocessing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/scripts_postprocessing.py).

9. **Delivery** — [`modules/infotext_utils.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/infotext_utils.py) packages metadata into the image, and the result returns to the UI or API client.

## Working with the Pipeline: Code Examples

These practical examples demonstrate how to interact with the pipeline at different abstraction layers.

### Calling the REST API

The WebUI exposes the pipeline via REST endpoints defined in [`modules/api/api.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/api/api.py). This example generates an image using the `txt2img` endpoint:

```python
import requests, json, base64

url = "http://127.0.0.1:7860/sdapi/v1/txt2img"

payload = {
    "prompt": "a cyberpunk city at sunset, ultra‑realistic",
    "negative_prompt": "lowres, blurry",
    "steps": 30,
    "cfg_scale": 7.5,
    "sampler_name": "Euler a",
    "width": 768,
    "height": 512,
    "seed": -1,
    "batch_size": 1,
    "n_iter": 1
}

r = requests.post(url, json=payload)
result = r.json()

with open("out.png", "wb") as f:
    f.write(base64.b64decode(result["images"][0]))

```

### Using the Internal Processing Class

For programmatic control within the Python environment, instantiate the `StableDiffusionProcessingTxt2Img` class directly:

```python
from modules import txt2img, processing, shared

class DummyRequest: pass
req = DummyRequest()

p = processing.StableDiffusionProcessingTxt2Img(
    sd_model=shared.sd_model,
    outpath_samples=shared.outpath_samples,
    prompt="portrait of a young woman, oil painting",
    negative_prompt="watermark",
    steps=20,
    cfg_scale=8,
    sampler_name="DPM++ 2M Karras",
    width=512,
    height=512,
    seed=123456789,
    batch_size=1,
    n_iter=1,
)

images = txt2img.txt2img(p, req)
print(f"Generated {len(images)} image(s)")

```

### Creating Custom Post-Processing Scripts

Extend the pipeline by inheriting from `ScriptPostprocessing` in [`modules/scripts_postprocessing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/scripts_postprocessing.py):

```python
import scripts.postprocessing_upscale as base

class ScriptPostprocessingMyUpscale(base.ScriptPostprocessingUpscale):
    def title(self):
        return "My Custom Upscale"

    def show(self, is_img2img):
        return not is_img2img

```

Place this file in the `scripts/` directory; the WebUI automatically discovers classes derived from `ScriptPostprocessing` and adds them to the Post-Processing panel.

## Summary

The AUTOMATIC1111 Stable Diffusion WebUI processes image generation through a rigorously modular pipeline:

- **Front-end abstraction** in [`webui.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/webui.py) and [`modules/api/api.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/api/api.py) queues jobs and validates inputs
- **Processing objects** in [`modules/processing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/processing.py) encapsulate all generation parameters
- **SD-Hijack** layers in [`modules/sd_hijack.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_hijack.py) inject CFG and prompt weighting into UNet and CLIP forward passes
- **Prompt parsing** in [`modules/prompt_parser.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/prompt_parser.py) handles tokenization and conditioning vector generation
- **Sampler loops** in [`modules/sd_samplers.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_samplers.py) orchestrate the denoising process using hijacked model calls
- **VAE decoding** in [`modules/sd_vae.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_vae.py) converts latents to RGB pixels
- **Safety and watermarking** in [`modules/safe.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/safe.py) filter content and embed provenance data
- **Post-processing** via [`modules/scripts_postprocessing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/scripts_postprocessing.py) enables upscaling and restoration
- **Metadata packaging** in [`modules/infotext_utils.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/infotext_utils.py) embeds generation parameters into the final image

This decoupled architecture allows developers to replace any component—such as plugging in a custom sampler or VAE—without destabilizing the entire pipeline.

## Frequently Asked Questions

### What is the SD-Hijack layer and why is it necessary?

The **SD-Hijack** layer, implemented in [`modules/sd_hijack.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_hijack.py) and its variants, replaces the native `forward` methods of the UNet, VAE, and CLIP models with wrapped versions. This interception is necessary to inject **classifier-free guidance (CFG)** logic, handle prompt weighting syntax like `(word:1.2)`, and enable custom attention mechanisms without modifying the underlying Stable Diffusion model weights. Without this layer, the WebUI could not support advanced features like negative prompts or dynamic CFG scaling during the sampling loop.

### How does the WebUI handle different samplers?

The WebUI abstracts all samplers through [`modules/sd_samplers.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_samplers.py) and backend-specific implementations like [`modules/sd_samplers_kdiffusion.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_samplers_kdiffusion.py). Each sampler implements a standardized interface that accepts the current latent tensor, timestep, and conditioning vectors, then returns the predicted noise. During the diffusion loop, the sampler repeatedly calls the **hijacked UNet** via its wrapped `forward` method, allowing the WebUI to switch between Euler, DDIM, DPM++, or custom samplers without changing the core model inference code.

### Where is the generation metadata stored?

Generation metadata—including the prompt, seed, CFG scale, and sampler name—is assembled by [`modules/infotext_utils.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/infotext_utils.py) into an **infotext** string. This string is embedded into the PNG file's metadata chunks (or returned in the API response) and displayed in the UI's generation info panel. This allows users to reproduce images exactly by reading the embedded parameters back into the WebUI.

### Can I modify the pipeline without editing core files?

Yes. The architecture supports extension through several mechanisms:
- **Post-processing scripts:** Inherit from `ScriptPostprocessing` in [`modules/scripts_postprocessing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/scripts_postprocessing.py) and place files in the `scripts/` directory to add upscaling or restoration steps.
- **Custom samplers:** Implement the sampler interface in [`modules/sd_samplers.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/sd_samplers.py) to add new diffusion algorithms.
- **Script extensions:** Use the `Script` base class to inject code at specific pipeline hooks (pre-process, post-process, etc.) without modifying [`webui.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/webui.py) or [`modules/processing.py`](https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/main/modules/processing.py).

This modular design ensures that custom logic integrates cleanly without destabilizing the core **image generation processing pipeline**.