How Memory Management Works in AUTOMATIC1111’s Low VRAM Mode

AUTOMATIC1111’s Web UI reduces GPU memory usage by activating a module-swapping system that keeps only one heavy neural network component in VRAM at a time, moving the rest to CPU memory during inference.

AUTOMATIC1111/stable-diffusion-webui enables users with limited graphics hardware to run large diffusion models through sophisticated memory management optimizations. When launched with --lowvram or --medvram flags, the application restructures model loading and execution in modules/lowvram.py to minimize peak VRAM consumption. This system allows GPUs with as little as 2 GB of memory to generate images by serializing access to model components rather than loading everything simultaneously.

Detecting Low VRAM Mode via Command-Line Flags

The activation sequence begins in modules/cmd_args.py, which defines the --lowvram, --medvram, and --medvram_sdxl arguments. During model loading in modules/sd_models.py, the function lowvram.is_needed() checks these flags:

def is_needed(sd_model):
    return shared.cmd_opts.lowvram or shared.cmd_opts.medvram or \
           shared.cmd_opts.medvram_sdxl and hasattr(sd_model, 'conditioner')

If any flag is active, is_needed returns True, triggering the low VRAM preparation pipeline at line 749 of modules/sd_models.py where lowvram.apply(m) is invoked.

Initializing the Module Swapping System

The apply() function in modules/lowvram.py performs two critical initialization steps. First, it disables parallel processing because operations must be serialized when components cannot reside on the GPU simultaneously:

shared.parallel_processing_allowed = not enable

Second, it calls setup_for_low_vram(sd_model, not shared.cmd_opts.lowvram) to restructure the model's memory layout. This function marks the model with sd_model.lowvram = True and identifies large components—such as first_stage_model, cond_stage_model, depth_model, embedder, and the main model—that should remain in CPU memory until needed.

The CPU-Offload Preparation Process

Inside setup_for_low_vram, the code temporarily detaches these heavy sub-modules, moves the model skeleton to the GPU, then re-attaches the components in CPU memory. This ensures that only the lightweight structural elements occupy VRAM initially, while the parameter-heavy weights stay on the system RAM:

• Lines 83-94 handle architecture-specific variations for SDXL or SD 2.0, adding conditioner modules to the CPU-resident list when appropriate.
• Lines 97-107 execute the detach-move-reattach sequence that establishes the split memory layout.

The Forward Hook That Swaps Modules on Demand

The core memory management mechanism is the send_me_to_gpu forward pre-hook registered on every heavy sub-module. Before any module executes, this hook moves it from CPU to GPU and evacuates the previous module:

def send_me_to_gpu(module, _):
    global module_in_gpu
    module = parents.get(module, module)
    if module_in_gpu == module:
        return
    if module_in_gpu is not None:
        module_in_gpu.to(cpu)                # move previous module back

    module.to(devices.device)                # move current module to GPU

    module_in_gpu = module

Located at lines 42-59 in modules/lowvram.py, this function maintains a global module_in_gpu reference to track which component currently resides in VRAM. When a new sub-module is invoked during the forward pass, the hook immediately swaps the tensors, ensuring only one heavy component occupies GPU memory at any moment.

Med VRAM and SDXL Optimizations

The system provides granular control for different hardware tiers through the medvram variants.

Med VRAM Mode: When --medvram is active, the UNet model itself receives additional hooking at lines 144-162. This allows individual sub-blocks of the UNet to swap independently rather than moving the entire UNet as one unit, further reducing peak memory for GPUs with 4-6 GB VRAM.

SDXL Support: For Stable Diffusion XL models, the code detects the conditioner attribute (lines 83-90) and registers the swap hook directly on the conditioning module at line 125. This accommodates SDXL's larger text encoder while maintaining the same single-module-in-VRAM invariant.

Runtime Control and Cleanup

The module provides utility functions for querying and manipulating the low VRAM state during execution.

Checking Status: lowvram.is_enabled(sd_model) (lines 64-66) returns the boolean flag stored on the model instance, allowing other components like modules/processing.py to select memory-appropriate code paths.

Manual Evacuation: send_everything_to_cpu() (lines 11-18) moves the currently resident GPU module back to CPU memory. This function is called throughout the codebase—such as in modules/sd_vae.py (line 266) and modules/processing.py (line 1009)—to clear VRAM before loading VAE variants or starting new sampling operations.

Practical Implementation Examples

Launch the Web UI with low VRAM optimizations using the command-line flag:

python webui.py --lowvram

Dynamically toggle low VRAM mode at runtime for an already-loaded model:

from modules import lowvram, shared

if not lowvram.is_enabled(shared.sd_model):
    lowvram.apply(shared.sd_model)    # activate low VRAM

else:
    lowvram.send_everything_to_cpu()  # clear VRAM, return to normal mode

Inspect which module currently resides in GPU memory:

from modules.lowvram import module_in_gpu
print("Active GPU module:", module_in_gpu)

Register custom heavy modules (such as experimental VAEs) for automatic swapping:

from modules.lowvram import send_me_to_gpu
my_custom_module.register_forward_pre_hook(send_me_to_gpu)

Summary

• Command-line flags (--lowvram, --medvram) in modules/cmd_args.py trigger the memory management system during model loading.
• setup_for_low_vram in modules/lowvram.py splits the model between CPU and GPU, keeping heavy weights in system RAM.
• send_me_to_gpu acts as a forward pre-hook that swaps modules on demand, maintaining only one heavy component in VRAM at any time.
• Med VRAM and SDXL modes provide architecture-specific hooking strategies for the UNet blocks and conditioning modules.
• Runtime utilities like is_enabled() and send_everything_to_cpu() allow the inference pipeline to query status and manually free VRAM.

Frequently Asked Questions

What is the difference between --lowvram and --medvram in AUTOMATIC1111?

--lowvram moves the entire UNet, VAE, and text encoders to CPU memory, swapping only one module at a time during inference, suitable for GPUs with 2-4 GB VRAM. --medvram keeps the UNet structure in GPU memory but hooks individual UNet sub-blocks for granular swapping, targeting GPUs with 4-6 GB VRAM where full offloading is unnecessary.

How does the module swapping mechanism affect generation speed?

The forward pre-hook introduces CPU-GPU transfer overhead because tensors must stream across the PCIe bus for each forward pass. Consequently, low VRAM mode reduces peak memory usage at the cost of slower inference, as the system cannot parallelize operations or keep all model weights resident simultaneously.

Can I enable low VRAM mode after the model has already loaded?

Yes. The lowvram.apply() function can be called dynamically on an existing model instance, as implemented in modules/lowvram.py. This restructures the model's memory layout and installs the send_me_to_gpu hooks without requiring a Web UI restart, though you should call lowvram.send_everything_to_cpu() first to clear existing allocations.

Where does the Web UI check if low VRAM is active during sampling?

The inference pipeline in modules/processing.py queries lowvram.is_enabled(shared.sd_model) before selecting sampler configurations. Additionally, modules/sd_samplers_common.py disables full live previews when low VRAM is active to prevent out-of-memory errors during the sampling loop.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}