# How the Image-to-Image (img2img) Process Works in Stable Diffusion: Pipeline Mechanics and the Strength Parameter

> Explore the Stable Diffusion img2img process. Learn how latent space encoding, noise steps, and the strength parameter control image generation by balancing original structure and prompt adherence. Optimize your creative workflow.

- Repository: [CompVis - Computer Vision and Learning LMU Munich/stable-diffusion](https://github.com/CompVis/stable-diffusion)
- Tags: deep-dive
- Published: 2026-03-01

---

**The image-to-image process in Stable Diffusion encodes an input image into latent space, uses the `strength` parameter to determine exactly how many forward diffusion noise steps to apply (calculated as `t_enc = int(strength * ddim_steps)`), and then runs reverse diffusion conditioned on a text prompt to generate a new image that balances original structure against prompt adherence.**

The image-to-image (img2img) process in Stable Diffusion allows you to transform existing images while preserving their underlying structure by manipulating their latent representations. Implemented in the CompVis/stable-diffusion repository, this pipeline leverages the latent diffusion model’s variational autoencoder (VAE) to inject a controlled amount of noise into a source image before running guided reverse diffusion. Understanding the mechanics behind the `strength` parameter and the latent encoding process is essential for controlling the balance between image fidelity and prompt adherence.

## The Image-to-Image Pipeline: From Pixels to Latents

The img2img workflow implemented in [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py) consists of seven distinct stages that transform an input image through the latent diffusion process.

### Step 1: Loading and Preprocessing the Input Image

The pipeline begins by loading the source image and preparing it for the VAE encoder. In [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py), the `load_img` function (lines 48-57) handles this by converting the image to RGB, resizing dimensions to the nearest multiple of 32 pixels (required by the convolutional VAE), and normalizing pixel values to the range `[-1, 1]`.

### Step 2: Encoding to Latent Space

Once preprocessed, the image tensor passes through the first-stage VAE defined in [`ldm/models/autoencoder.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/autoencoder.py). The code calls `model.encode_first_stage(init_image)` followed by `model.get_first_stage_encoding()` (lines 34-36 in [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py)) to compress the image into a latent tensor `z` with significantly reduced dimensionality. This latent representation captures the essential structure of the original image while discarding high-frequency noise.

### Step 3: Calculating Noise Steps with the Strength Parameter

The `strength` parameter (defined in lines 60-64 of [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py)) serves as the critical control knob for the img2img process. The implementation calculates the number of forward diffusion steps to apply using the formula:

```python
t_enc = int(strength * opt.ddim_steps)

```

Here, `strength` is a float in the range `[0, 1]`, and `opt.ddim_steps` represents the total number of DDIM sampling steps. This calculation (lines 39-42 in [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py)) determines exactly how much of the original latent information will be destroyed by noise.

### Step 4: Stochastic Encoding (Adding Diffusion Noise)

With `t_enc` calculated, the pipeline calls `sampler.stochastic_encode()` from [`ldm/models/diffusion/ddim.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/diffusion/ddim.py) (lines 58-60 in [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py)). This method applies the forward diffusion process to the clean latent `z` for exactly `t_enc` timesteps, producing a noisy latent `z_enc`. The amount of noise injected directly correlates with the `strength` value—higher strength means more forward steps, resulting in a latent that resembles pure Gaussian noise and retains less original image structure.

### Step 5: Reverse Diffusion and Decoding

The sampler then executes the reverse diffusion process starting from the noisy `z_enc`. The `sampler.decode()` method (lines 61-62 in [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py)) runs for the remaining `ddim_steps - t_enc` steps, guided by the text prompt conditioning `c` and the unconditional (null) prompt `uc` for classifier-free guidance. The `scale` parameter controls guidance strength during this phase, but the `strength` parameter has already determined the starting point of this reverse journey.

### Step 6: Decoding Back to Pixel Space

After the sampler produces the final latent representation, the pipeline calls `model.decode_first_stage()` (lines 64-66 in [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py)) from [`ldm/models/autoencoder.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/autoencoder.py) to convert the latent tensor back into pixel space. The output is clamped to the `[0, 1]` range and converted to a PIL Image for saving.

## What the Strength Parameter Controls in img2img

The `strength` parameter acts as a linear interpolation controller between **image preservation** and **prompt-driven transformation**. When you set `strength=0.0`, the calculation `t_enc = int(0.0 * ddim_steps)` results in zero forward diffusion steps. In this edge case, `stochastic_encode` adds no noise, and the reverse diffusion starts from the clean original latent, effectively returning the input image (subject only to minor VAE reconstruction errors).

Conversely, when `strength=1.0`, the formula yields `t_enc = ddim_steps`, meaning the forward diffusion process runs for the full duration of the sampling schedule. This completely destroys the original image information, replacing it with pure Gaussian noise. The subsequent reverse diffusion then behaves exactly like standard text-to-image generation, producing results that ignore the input image structure entirely.

Practically, values between 0.3 and 0.7 provide the best balance, where `strength=0.5` applies noise for half the total steps, allowing the prompt to guide the composition while preserving the original layout and coarse features.

## Practical Code Examples for Stable Diffusion img2img

### Command-Line Interface Usage

The most common way to run img2img is through the provided script in [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py). Here is a typical invocation that demonstrates the `strength` parameter:

```bash
python scripts/img2img.py \
  --init-img path/to/input.png \
  --prompt "a painting of a cyberpunk city at sunset" \
  --strength 0.6 \
  --ddim_steps 50 \
  --scale 7.5 \
  --outdir outputs/img2img_example

```

In this example, `--strength 0.6` calculates `t_enc = 0.6 * 50 = 30` forward diffusion steps. This preserves approximately 40% of the original latent information while allowing the prompt to influence the remaining 60% of the generation process. Adjust `--ddim_steps` to change the total sampling budget; the effective noise level scales proportionally with `strength`.

### Programmatic Implementation in Python

For custom applications, you can implement the img2img pipeline directly using the underlying classes. This example mirrors the logic found in [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py):

```python
from ldm.models.diffusion.ddim import DDIMSampler
from ldm.util import instantiate_from_config
from omegaconf import OmegaConf
import torch
import numpy as np
from PIL import Image

# Load model configuration and weights

config = OmegaConf.load("configs/stable-diffusion/v1-inference.yaml")
model = instantiate_from_config(config.model).cuda().eval()
ckpt = torch.load("models/ldm/stable-diffusion-v1/model.ckpt", map_location="cpu")
model.load_state_dict(ckpt["state_dict"], strict=False)

# Initialize the DDIM sampler

sampler = DDIMSampler(model)

# Load and preprocess the input image

init_image = Image.open("input.png").convert("RGB")
w, h = map(lambda x: x - x % 32, init_image.size)  # Ensure dimensions are multiples of 32

init_image = init_image.resize((w, h), resample=Image.LANCZOS)
init_array = (np.array(init_image).astype(np.float32) / 255.0)[None].transpose(0, 3, 1, 2)
init_tensor = torch.from_numpy(init_array).to("cuda") * 2.0 - 1.0

# Encode to latent space

init_latent = model.get_first_stage_encoding(model.encode_first_stage(init_tensor))

# Configure diffusion parameters

ddim_steps = 50
strength = 0.75
t_enc = int(strength * ddim_steps)  # Number of forward diffusion steps

# Apply stochastic encoding (forward diffusion)

z_enc = sampler.stochastic_encode(init_latent, torch.tensor([t_enc]).to("cuda"))

# Prepare text conditioning

c = model.get_learned_conditioning(["a cyberpunk city at sunset"])
uc = model.get_learned_conditioning([""])  # Unconditional for classifier-free guidance

# Run reverse diffusion

samples = sampler.decode(z_enc, c, t_enc,
                        unconditional_guidance_scale=7.5,
                        unconditional_conditioning=uc)

# Decode to pixel space

output = model.decode_first_stage(samples)
output = torch.clamp((output + 1) / 2, 0, 1)
output_image = Image.fromarray((output[0].cpu().numpy().transpose(1, 2, 0) * 255).astype(np.uint8))
output_image.save("output.png")

```

This implementation explicitly shows how `strength` determines `t_enc`, which controls the number of forward diffusion steps applied via `stochastic_encode` before the reverse diffusion process begins.

## Key Source Files in the CompVis/stable-diffusion Repository

Understanding the img2img implementation requires familiarity with several critical files that handle encoding, sampling, and configuration:

- **[`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py)**: The main entry point that orchestrates the entire pipeline. It parses the `strength` argument, calculates `t_enc`, and coordinates calls to `stochastic_encode` and `decode` (lines 34-66).

- **[`ldm/models/diffusion/ddim.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/diffusion/ddim.py)**: Implements the `DDIMSampler` class, providing the `stochastic_encode` method for forward diffusion and the `decode` method for reverse diffusion sampling.

- **[`ldm/models/autoencoder.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/autoencoder.py)**: Contains the first-stage VAE used by `encode_first_stage` and `decode_first_stage` to convert between pixel space and latent representations.

- **[`ldm/util.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/util.py)**: Provides helper functions like `instantiate_from_config` used to load model configurations.

- **[`configs/stable-diffusion/v1-inference.yaml`](https://github.com/CompVis/stable-diffusion/blob/main/configs/stable-diffusion/v1-inference.yaml)**: Defines the model architecture including the U-Net, VAE, and text encoder configurations required for the img2img pipeline.

## Summary

The image-to-image process in Stable Diffusion transforms existing images by manipulating their latent representations through a controlled noise injection mechanism. Key takeaways include:

- The pipeline encodes input images into latents using the first-stage VAE (`encode_first_stage`), applies forward diffusion noise via `stochastic_encode`, and reconstructs the image through reverse diffusion (`decode`) and VAE decoding (`decode_first_stage`).

- The `strength` parameter (range 0.0 to 1.0) directly determines the number of forward diffusion steps (`t_enc = int(strength * ddim_steps)`), controlling the trade-off between preserving the original image structure and allowing prompt-driven transformation.

- A `strength` of 0.0 preserves the original image (zero noise added), while 1.0 completely destroys the input information, making the process equivalent to standard text-to-image generation.

## Frequently Asked Questions

### What happens when I set strength to 0.0 or 1.0 in img2img?

When `strength` is set to `0.0`, the calculation `t_enc = int(0.0 * ddim_steps)` results in zero forward diffusion steps. The `stochastic_encode` function adds no noise to the latent, and the reverse diffusion starts from the clean original image, effectively returning the input image unchanged (subject only to minor VAE reconstruction artifacts). Conversely, when `strength` is `1.0`, `t_enc` equals the total number of DDIM steps, meaning the forward diffusion process runs to completion and completely destroys the original image information, making the output equivalent to pure text-to-image generation that ignores the input structure.

### How does the strength parameter affect the number of diffusion steps?

The `strength` parameter acts as a multiplier against the total sampling budget defined by `ddim_steps`. The formula `t_enc = int(strength * ddim_steps)` (implemented in [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py), lines 39-42) determines how many of the total steps are allocated to the forward (noise-adding) process versus the reverse (denoising) process. For example, with `ddim_steps=50` and `strength=0.6`, the system applies 30 steps of forward diffusion to destroy latent information, then uses the remaining 20 steps for reverse diffusion guided by the prompt. Higher strength values allocate more steps to destruction, leaving fewer steps for reconstruction, which results in greater deviation from the original image.

### Why must image dimensions be multiples of 32 in the img2img pipeline?

The dimension requirement stems from the architecture of the first-stage VAE (variational autoencoder) defined in [`ldm/models/autoencoder.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/autoencoder.py). The encoder and decoder use convolutional layers with downsampling and upsampling operations that reduce spatial dimensions by factors of 2 (typically 64x64 or 32x32 latent representations for 512x512 images). To ensure that the spatial dimensions remain integers throughout the encoding and decoding process without padding artifacts or dimension mismatches, the input image width and height must be divisible by 32 (or more specifically, by the downsampling factor of the VAE, which is typically 8 for 512→64 compression, but the code in [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py) uses 32 to be safe for various configurations).

### What is the difference between the strength and scale parameters in img2img?

While both parameters influence the final output, they control fundamentally different aspects of the generation process. The `strength` parameter (sometimes called denoising strength) controls how much of the original image is preserved by determining the number of forward diffusion steps applied to the latent representation via `stochastic_encode`. It ranges from 0.0 (complete preservation) to 1.0 (complete replacement). In contrast, the `scale` parameter (classifier-free guidance scale) controls the influence of the text prompt during the reverse diffusion process executed by `sampler.decode`. Higher scale values force the generation to adhere more strictly to the prompt at the cost of image quality or diversity, but unlike strength, scale does not determine how much of the original image information is retained—it only affects how the denoising is guided.