AUTOMATIC1111 Upscalers Explained: ESRGAN, RealESRGAN, and CodeFormer Differences

TLDR: AUTOMATIC1111's Stable Diffusion WebUI provides three categories of upscalers: classic resampling filters (Lanczos, Nearest), deep learning models (ESRGAN, RealESRGAN, HAT, SwinIR), and post-processing face restoration (CodeFormer), each implemented in specific modules from modules/upscaler.py to modules/realesrgan_model.py and scripts/postprocessing_codeformer.py.

The AUTOMATIC1111 Stable Diffusion WebUI ships with a comprehensive upscaling ecosystem that ranges from simple Pillow filters to sophisticated neural networks. Understanding the differences between ESRGAN, RealESRGAN, and CodeFormer—alongside other available upscalers—helps you optimize image quality based on your hardware constraints and content type. This guide examines the source implementation in the AUTOMATIC1111/stable-diffusion-webui repository to explain how each upscaler works and when to use it.

Three Categories of Upscalers in AUTOMATIC1111

According to the source code in modules/upscaler.py and its specialized implementations, upscalers available in AUTOMATIC1111 fall into three distinct architectures:

• Classic resampling: Fast Pillow-based filters requiring no model downloads or GPU inference
• Neural-network models: Deep learning approaches using Spandrel-wrapped GANs and Transformers for detail reconstruction
• Post-processing enhancement: Face-specific restoration that operates after the primary upscaling stage

Classic Resampling Upscalers (None, Lanczos, Nearest)

For users prioritizing speed over AI-enhanced detail, the WebUI includes three basic resampling options defined in modules/upscaler.py.

Lanczos (modules/upscaler.py, line 123) uses Pillow’s Lanczos filter to provide high-quality interpolation for any integer scale factor, commonly 2× to 4×. This method delivers good results for simple upscales without consuming VRAM for model inference.

Nearest (modules/upscaler.py, line 138) applies Pillow’s nearest-neighbor filter, producing a fast but blocky "pixel-art" aesthetic suitable for specific artistic styles or retro graphics.

None (modules/upscaler.py, line 108) acts as a pass-through when you want to perform downstream post-processing without any initial scaling operation.

Deep Learning Upscalers: ESRGAN and RealESRGAN

The WebUI’s neural upscalers leverage the Spandrel library to wrap pre-trained PyTorch models, offering superior detail reconstruction compared to classic methods.

ESRGAN Implementation

ESRGAN (modules/esrgan_model.py) provides sharp, detail-preserving upscaling at a fixed 4× scale defined by the model architecture. The implementation downloads ESRGAN_4x.pth on demand and excels at photorealistic content, though it may amplify artifacts in heavily compressed sources.

RealESRGAN Differences

Real-ESRGAN (modules/realesrgan_model.py) extends the ESRGAN architecture with a degradation-aware pipeline that handles broader content types including photos and anime. Unlike vanilla ESRGAN's fixed 4× output, Real-ESRGAN supports both 2× and 4× scales through multiple packaged models configured in get_realesrgan_models(). Real-ESRGAN specifically reduces ringing artifacts and blur common in real-world photographs, making it more robust for general use than standard ESRGAN.

Additional Neural Options

Beyond ESRGAN variants, the repository includes specialized architectures for specific use cases:

• HAT (modules/hat_model.py): Implements a Highly-Advanced-Texture network for 4× upscaling, optimized for high-frequency details like fabric and surfaces
• SwinIR (extensions-builtin/SwinIR/scripts/swinir_model.py): Uses Swin-Transformer blocks for strong natural image performance at configurable 2×/4× scales
• ScuNET (extensions-builtin/ScuNET/scripts/scunet_model.py): Delivers faster convolutional-fusion processing than SwinIR with modest quality trade-offs
• LDSR (extensions-builtin/LDSR/scripts/ldsr_model.py): Provides lightweight deep super-resolution with minimal VRAM requirements for low-end GPUs

CodeFormer: Face Restoration vs. Upscaling

CodeFormer operates differently than traditional upscalers. Located in scripts/postprocessing_codeformer.py, it functions as a post-processing enhancement stage rather than a scaling algorithm. While standard upscalers enlarge the entire image, CodeFormer uses a facial restoration auto-encoder to improve portrait fidelity and remove artifacts at 1× scale.

When used in the Extras tab, CodeFormer blends its restored face output with the previously upscaled image using a configurable visibility slider (0.0 to 1.0) and weight parameter (typically 0.0 to 1.0). The actual enlargement is performed by your selected upscaler (such as Real-ESRGAN), while CodeFormer handles subsequent facial refinement.

Configuring Upscalers in the Web UI

To apply these upscalers through the interface:

1. Navigate to the Extras tab and select Upscale
2. Set your desired Scale factor (2×, 4×, or custom integers for classic methods)
3. Choose an upscaler from the dropdown populated by modules/esrgan_model.py and modules/realesrgan_model.py
4. For face-heavy images, enable CodeFormer in the accordion below and adjust the visibility and weight sliders
5. Click Generate to execute the pipeline

Programmatic Upscaler Usage

You can also trigger upscalers programmatically using the WebUI’s Python API:

from modules import upscaler
from PIL import Image

# Initialize Real-ESRGAN from modules/realesrgan_model.py

real_esrgan = upscaler.UpscalerRealESRGAN(path="models/RealESRGAN")
real_esrgan.__init__(path="models/RealESRGAN")
selected = real_esrgan.scalers[0].data_path

img = Image.open("input.png")
upscaled = real_esrgan.upscale(img, scale=4, selected_model=selected)
upscaled.save("output.png")

For CodeFormer post-processing:

from scripts.postprocessing_codeformer import ScriptPostprocessingCodeFormer
from modules import scripts_postprocessing
from PIL import Image

pp = scripts_postprocessing.PostprocessedImage(image=Image.open("upscaled.png"), info={})
codeformer = ScriptPostprocessingCodeFormer()

# Apply face restoration with 80% visibility

codeformer.process(pp, enable=True, codeformer_visibility=0.8, codeformer_weight=0.3)
pp.image.save("restored.png")

Summary

• Classic resampling in modules/upscaler.py offers fast, no-model solutions via Lanczos and Nearest filters for any integer scale
• ESRGAN (modules/esrgan_model.py) delivers fixed 4× photorealistic upscaling, while Real-ESRGAN (modules/realesrgan_model.py) adds degradation-aware processing for 2× and 4× outputs with better artifact suppression
• CodeFormer (scripts/postprocessing_codeformer.py) performs 1× face restoration as a post-processing blend, not as a primary upscaler
• Alternative neural models including HAT, SwinIR, ScuNET, and LDSR cater to specific texture detail, speed, or low-memory requirements

Frequently Asked Questions

What is the difference between ESRGAN and RealESRGAN in AUTOMATIC1111?

ESRGAN, implemented in modules/esrgan_model.py, provides fixed 4× upscaling optimized for sharp detail preservation using standard GAN training on DIV2K datasets. Real-ESRGAN, found in modules/realesrgan_model.py, introduces a degradation-aware pipeline that handles real-world photos and anime more robustly, offering both 2× and 4× models while suppressing compression artifacts better than vanilla ESRGAN.

Does CodeFormer upscale images or just restore faces?

CodeFormer does not upscale images; it operates at 1× as a post-processing face enhancer defined in scripts/postprocessing_codeformer.py. When you select CodeFormer in the Extras tab, the WebUI first applies your chosen upscaler (such as Real-ESRGAN or Lanczos) to enlarge the image, then CodeFormer blends restored facial details using its visibility and weight parameters.

Which upscaler is best for low VRAM GPUs?

LDSR (extensions-builtin/LDSR/scripts/ldsr_model.py) provides the lowest memory footprint among neural options, while Lanczos (modules/upscaler.py) requires virtually no VRAM since it uses CPU-based Pillow resampling. For deep learning upscaling on constrained hardware, ScuNET offers a faster alternative to SwinIR with modest quality trade-offs.

Where are upscaler model files stored in AUTOMATIC1111?

Neural upscaler checkpoints reside in subdirectories under models/: ESRGAN and HAT models live in models/ESRGAN/, Real-ESRGAN downloads to models/RealESRGAN/, while extension-specific models like SwinIR, ScuNET, and LDSR store their .pth files within their respective extensions-builtin/ subfolders. Classic resamplers require no external model files.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}