How to Implement Custom Upscaling with ESRGAN Models in ComfyUI

To implement custom upscaling with ESRGAN models in ComfyUI, place your checkpoint in the models/upscale_models/ directory, load it using the Load Upscale Model node, and process images through the Upscale Image (with Model) node which automatically handles tiling and VRAM management.

ComfyUI provides a generic upscaling pipeline that works with any single-image super-resolution model following the spandrel ImageModelDescriptor API, including ESRGAN, Real-ESRGAN, and SwinIR variants. This guide explains how to implement custom upscaling with ESRGAN models using the built-in nodes in the Comfy-Org/ComfyUI repository.

Understanding the Upscale Pipeline Architecture

The upscaling system consists of two primary nodes that handle model loading and inference execution.

Core Nodes Overview

The pipeline relies on two nodes defined in comfy_extras/nodes_upscale_model.py:

• Load Upscale Model: Loads ESRGAN-style checkpoints from folder_paths["upscale_models"] and returns a spandrel.ImageModelDescriptor object
• Upscale Image (with Model): Executes the model on input tensors, handling tiling, VRAM-aware OOM fallback, and post-processing

Spandrel Integration

ComfyUI uses the spandrel library to abstract model architectures. When you implement custom upscaling with ESRGAN models, the system expects checkpoints compatible with spandrel.ImageModelDescriptor. The ModelLoader().load_from_state_dict() function handles architecture detection automatically.

Preparing Your ESRGAN Model

Before loading a custom model, you must place it in a directory indexed by ComfyUI's path system.

Default Model Directory

By default, ComfyUI looks for upscaling models in models/upscale_models/. This path is defined in folder_paths.py:

folder_names_and_paths["upscale_models"] = ([os.path.join(models_dir, "upscale_models")], supported_pt_extensions)

Copy or symlink your ESRGAN .safetensors, .pth, or .ckpt files into this directory:

ComfyUI/
└─ models/
   └─ upscale_models/
      ├─ ESRGAN_x4.safetensors
      └─ RealESRGAN_x4plus.safetensors

External Model Paths Configuration

If you maintain models in a central repository outside the ComfyUI tree, expose the path via extra_model_paths.yaml. Rename the provided extra_model_paths.yaml.example and add your directories:

# extra_model_paths.yaml

upscale_models:
  - /path/to/central/ESRGAN
  - /path/to/RealESRGAN

Save the file in the ComfyUI root and restart the application so the new paths are added to folder_paths["upscale_models"].

Loading and Executing the Model

Once your checkpoint is accessible, use the built-in nodes to process images.

The Load Upscale Model Node

In the ComfyUI editor:

1. Add a Load Upscale Model node from the loaders category
2. Select your checkpoint from the model_name dropdown (populated by folder_paths.get_filename_list("upscale_models"))
3. The node outputs an Upscale Model object containing the spandrel.ImageModelDescriptor and the model's native scale factor

The node executes this logic from comfy_extras/nodes_upscale_model.py (lines 35-44):

model_path = folder_paths.get_full_path_or_raise("upscale_models", model_name)
sd = comfy.utils.load_torch_file(model_path, safe_load=True)
out = ModelLoader().load_from_state_dict(sd).eval()
if not isinstance(out, ImageModelDescriptor):
    raise Exception("Upscale model must be a single-image model.")
return (out,)

The Upscale Image (with Model) Node

To process images:

1. Add an Upscale Image (with Model) node from the image/upscaling category
2. Connect the Upscale Model output to the node's upscale_model input
3. Connect an IMAGE tensor to the image input
4. Execute the workflow

Automatic Tiling and Memory Management

The ImageUpscaleWithModel node handles technical complexities automatically:

• VRAM-aware allocation: Calculates required memory and frees VRAM using model_management.free_memory before inference
• Tiling strategy: Splits images into 512px tiles with 32px overlap to prevent OOM errors
• OOM fallback: Automatically reduces tile size if CUDA runs out of memory
• Scale respect: Uses upscale_model.scale to determine output dimensions (typically 4× for ESRGAN)

The tiling logic (lines 78-92 in nodes_upscale_model.py) ensures you can run large ESRGAN models even on GPUs with limited VRAM.

Programmatic Implementation

For scripted workflows, invoke the nodes directly in Python:

import torch
from comfy import folder_paths, model_management, utils
from comfy_extras.nodes_upscale_model import UpscaleModelLoader, ImageUpscaleWithModel

# 1. Load the model

model_name = "ESRGAN_x4.safetensors"
model_path = folder_paths.get_full_path_or_raise("upscale_models", model_name)
state = utils.load_torch_file(model_path, safe_load=True)
upscale_model = UpscaleModelLoader().execute(model_name)[0]

# 2. Load an image (as torch tensor, shape [1, H, W, 3])

img = utils.load_image("input.png")

# 3. Upscale

out = ImageUpscaleWithModel().execute(upscale_model, img)[0]
utils.save_image(out, "upscaled.png")

This script mirrors the UI implementation exactly, including automatic tiling and VRAM management.

Troubleshooting Custom Models

Issue	Solution
Model not recognized (error "Upscale model must be a single-image model")	Ensure the checkpoint contains a single-image spandrel descriptor. For custom architectures, create a wrapper class inheriting from spandrel.ImageModelDescriptor and register it via spandrel_extra_arches following the pattern in nodes_upscale_model.py.
Different upscaling factor (e.g., 2×, 3×)	Check upscale_model.scale to verify the model's native factor. The node respects this value automatically, but you can query it programmatically if you need to adjust downstream processing.
Want to change tile size	The default is 512px with 32px overlap. To adjust globally, modify tile = 512 and overlap = 32 in ImageUpscaleWithModel.execute within comfy_extras/nodes_upscale_model.py.
Batch upscaling	Feed a batch of images with shape [N, H, W, 3] into the node. The tiling logic processes each image in the batch automatically.

Summary

• Place your ESRGAN checkpoint in models/upscale_models/ or expose it via extra_model_paths.yaml and restart ComfyUI.
• Load the model using the Load Upscale Model node, which wraps the checkpoint in a spandrel.ImageModelDescriptor via comfy_extras/nodes_upscale_model.py.
• Process images through the Upscale Image (with Model) node, which automatically handles 512px tiling, 32px overlap, VRAM management, and OOM recovery.
• Script the workflow programmatically using UpscaleModelLoader and ImageUpscaleWithModel classes for automated pipelines.

Frequently Asked Questions

Where does ComfyUI look for ESRGAN model files?

ComfyUI searches the upscale_models folder defined in folder_paths.py, which defaults to ComfyUI/models/upscale_models/. You can add additional search paths by creating an extra_model_paths.yaml file in the ComfyUI root and specifying upscale_models entries pointing to external directories.

Why does my custom ESRGAN model throw "Upscale model must be a single-image model"?

This error occurs in comfy_extras/nodes_upscale_model.py when the loaded checkpoint does not produce a spandrel.ImageModelDescriptor instance. Ensure your model follows the single-image super-resolution format. For custom architectures, you must create a wrapper class inheriting from ImageModelDescriptor and register it via the spandrel extra architectures system.

How does ComfyUI handle large images that don't fit in GPU memory?

The ImageUpscaleWithModel node implements automatic tiling with a default size of 512px and 32px overlap. If CUDA runs out of memory, the node catches the OOM exception and retries with smaller tiles. This logic in comfy_extras/nodes_upscale_model.py allows ESRGAN models to run on GPUs with as little as 1GB VRAM.

Can I use ESRGAN models with different scale factors like 2× or 3×?

Yes. The upscale_model.scale attribute stores the model's native upscaling factor, which the ImageUpscaleWithModel node respects automatically. When loading a model, the UpscaleModelLoader extracts this value from the checkpoint metadata. You can verify the scale factor by checking the model output or querying upscale_model.scale in a custom script.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}