# How to Optimize Memory Usage for Stable Diffusion Inference on Low-VRAM GPUs

> Run Stable Diffusion on low VRAM GPUs with half-precision weights, gradient checkpointing, and mixed precision. Optimize memory for efficient inference.

- Repository: [CompVis - Computer Vision and Learning LMU Munich/stable-diffusion](https://github.com/CompVis/stable-diffusion)
- Tags: performance
- Published: 2026-03-02

---

**Enable half-precision weights, activate gradient checkpointing, and use automatic mixed precision to run Stable Diffusion inference on GPUs with as little as 1 GB of VRAM.**

The CompVis/stable-diffusion repository includes built-in mechanisms to dramatically reduce GPU memory consumption during inference. By combining precision casting, activation checkpointing, and careful batch sizing, you can run the full UNet architecture—which contains approximately 860 million parameters—on consumer hardware with severely limited video memory.

## Three Core Memory Optimization Techniques

The codebase provides three complementary strategies that work together to minimize VRAM usage. Each targets a different aspect of the inference pipeline, from model weights to intermediate activations.

### Enable Half-Precision (FP16) Inference

Storing model weights and activations in `torch.float16` instead of `torch.float32` reduces tensor memory footprint by approximately 50 percent. In [`ldm/modules/diffusionmodules/openaimodel.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/modules/diffusionmodules/openaimodel.py), the `OpenAIModel` class accepts a `use_fp16` parameter that controls the model's internal dtype at line 500. When enabled, the UNet's parameter count drops from roughly 3.5 GB to 1.8 GB, fitting comfortably within constrained VRAM budgets alongside the latent space buffers.

### Activate Gradient Checkpointing

Gradient checkpointing trades computational overhead for memory savings by discarding intermediate feature maps after the forward pass and recomputing them only when necessary. The default inference configuration at [`configs/stable-diffusion/v1-inference.yaml`](https://github.com/CompVis/stable-diffusion/blob/main/configs/stable-diffusion/v1-inference.yaml) enables this via `use_checkpoint: true` at line 43. The actual implementation resides in [`ldm/modules/diffusionmodules/util.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/modules/diffusionmodules/util.py) (lines 102–117), where the `checkpoint` function wraps ResBlock and Attention operations to free activation caches immediately after use, rather than holding them for the entire sampling loop.

### Use Automatic Mixed Precision (Autocast)

The sampling scripts support a `--precision` flag that triggers `torch.autocast`, dynamically casting supported operations (convolutions, linear layers) to FP16 while preserving FP32 precision for numerically sensitive operations like softmax. In [`scripts/txt2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/txt2img.py), the autocast context manager is applied around the sampling loop at lines 28–33 and 88–90. This "best-of-both-worlds" approach delivers FP16 memory savings without requiring manual model conversion.

## Practical Implementation Guide

Combine these three techniques using either command-line flags or programmatic configuration overrides.

### Run Inference with Autocast and Checkpointing

The simplest approach uses the reference script with default optimizations:

```bash
python scripts/txt2img.py \
    --prompt "a tiny robot painting a sunrise" \
    --ckpt models/ldm/stable-diffusion-v1/model.ckpt \
    --config configs/stable-diffusion/v1-inference.yaml \
    --precision autocast \
    --n_samples 1 \
    --ddim_steps 50

```

The `--precision autocast` flag triggers the mixed-precision wrapper defined at lines 28–33 of [`scripts/txt2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/txt2img.py), while the default config already enables checkpointing via `use_checkpoint: true`.

### Force Full FP16 Mode Programmatically

For maximum memory reduction, explicitly configure the model to use half-precision weights:

```python
from omegaconf import OmegaConf
from ldm.util import instantiate_from_config
import torch

cfg = OmegaConf.load("configs/stable-diffusion/v1-inference.yaml")
cfg.model.params.use_fp16 = True        # Forces torch.float16 dtype

cfg.model.params.use_checkpoint = True  # Enables activation checkpointing

model = instantiate_from_config(cfg.model)
model = model.to(torch.device("cuda"))

```

The `use_fp16` parameter propagates to the UNet initialization in [`openaimodel.py`](https://github.com/CompVis/stable-diffusion/blob/main/openaimodel.py), setting the model's dtype to `th.float16` as implemented at line 500.

### Create a Custom Low-VRAM Configuration

Create a dedicated config file [`low_vram_config.yaml`](https://github.com/CompVis/stable-diffusion/blob/main/low_vram_config.yaml) for persistent optimization settings:

```yaml
model:
  target: ldm.modules.diffusionmodules.openaimodel.OpenAIModel
  params:
    image_size: 512
    in_channels: 4
    model_channels: 320
    out_channels: 4
    num_res_blocks: 2
    attention_resolutions: [4, 2, 1]
    use_checkpoint: true
    use_fp16: true

```

Then launch inference with this configuration:

```bash
python scripts/txt2img.py \
    --config low_vram_config.yaml \
    --precision autocast \
    --n_samples 1

```

## Monitoring Peak VRAM Consumption

Track actual memory allocation to verify optimization effectiveness using PyTorch's CUDA memory statistics:

```python
import torch

torch.cuda.reset_peak_memory_stats()

# ... run your inference loop ...

peak_mb = torch.cuda.max_memory_allocated() / (1024 ** 2)
print(f"Peak VRAM: {peak_mb:.1f} MiB")

```

This mirrors the reporting logic found in [`main.py`](https://github.com/CompVis/stable-diffusion/blob/main/main.py) at lines 398–413, which tracks peak memory usage during training and evaluation cycles.

## Summary

- **Half-precision storage** cuts model weight memory by 50 percent, reducing the 860-million-parameter UNet from ~3.5 GB to ~1.8 GB.
- **Gradient checkpointing** in [`ldm/modules/diffusionmodules/util.py`](https://github.com/CompVis/stable-diffusion/blob/main/ldm/modules/diffusionmodules/util.py) (lines 102–117) frees intermediate activation caches after each forward pass, preventing accumulation across the network depth.
- **Automatic mixed precision** via `--precision autocast` in [`scripts/txt2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/txt2img.py) applies dynamic FP16 casting without manual model conversion.
- **Batch size control** using `--n_samples 1` minimizes parallel allocation overhead during the latent diffusion process.
- Together, these settings enable 512×512 image generation on GPUs with approximately 1 GB of VRAM, typically staying under 900 MiB peak consumption.

## Frequently Asked Questions

### How much VRAM is required to run Stable Diffusion without optimizations?

Running the standard FP32 UNet with full precision and no checkpointing requires approximately 6–8 GB of VRAM for a batch size of one at 512×512 resolution. The 860-million-parameter model consumes roughly 3.5 GB for weights alone, with additional memory required for latent buffers, optimizer states, and intermediate activations.

### Does gradient checkpointing slow down inference speed?

Yes, gradient checkpointing increases computational overhead because intermediate activations must be recomputed during the backward pass or when needed for cross-attention guidance. However, for pure inference with classifier-free guidance, the impact is minimal—typically 10–20 percent slower per step—because the recomputation only occurs for attention layers, while the memory savings allow the model to run on hardware that would otherwise fail with out-of-memory errors.

### Can I use these optimizations with img2img or inpainting scripts?

Absolutely. The same memory optimization flags work for [`scripts/img2img.py`](https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py) and other sampling variants. Both scripts parse the `--precision` argument and apply the `torch.autocast` context manager around their respective sampling loops. Ensure you use `--n_samples 1` and the same YAML configuration with `use_checkpoint: true` to maintain low memory usage when processing existing images.

### Why does the default config enable checkpointing if it slows inference?

The [`v1-inference.yaml`](https://github.com/CompVis/stable-diffusion/blob/main/v1-inference.yaml) sets `use_checkpoint: true` at line 43 as a safety default to ensure broad hardware compatibility. The CompVis/stable-diffusion repository prioritizes accessibility across consumer GPUs—including those with 4–6 GB VRAM—over maximum inference speed. Users with abundant VRAM (12+ GB) can disable checkpointing by setting `use_checkpoint: false` in their configuration to trade memory for speed.