# How to Implement Custom Upscaling with ESRGAN Models in ComfyUI

> Learn to implement custom upscaling with ESRGAN models in ComfyUI. Easily load your models and upscale images efficiently, with automatic tiling and VRAM management.

- Repository: [Comfy Org/ComfyUI](https://github.com/Comfy-Org/ComfyUI)
- Tags: how-to-guide
- Published: 2026-02-26

---

**To implement custom upscaling with ESRGAN models in ComfyUI, place your checkpoint in the `models/upscale_models/` directory, load it using the Load Upscale Model node, and process images through the Upscale Image (with Model) node which automatically handles tiling and VRAM management.**

ComfyUI provides a generic upscaling pipeline that works with any single-image super-resolution model following the spandrel `ImageModelDescriptor` API, including ESRGAN, Real-ESRGAN, and SwinIR variants. This guide explains how to implement custom upscaling with ESRGAN models using the built-in nodes in the Comfy-Org/ComfyUI repository.

## Understanding the Upscale Pipeline Architecture

The upscaling system consists of two primary nodes that handle model loading and inference execution.

### Core Nodes Overview

The pipeline relies on two nodes defined in [`comfy_extras/nodes_upscale_model.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy_extras/nodes_upscale_model.py):

- **Load Upscale Model**: Loads ESRGAN-style checkpoints from `folder_paths["upscale_models"]` and returns a `spandrel.ImageModelDescriptor` object
- **Upscale Image (with Model)**: Executes the model on input tensors, handling tiling, VRAM-aware OOM fallback, and post-processing

### Spandrel Integration

ComfyUI uses the spandrel library to abstract model architectures. When you implement custom upscaling with ESRGAN models, the system expects checkpoints compatible with `spandrel.ImageModelDescriptor`. The `ModelLoader().load_from_state_dict()` function handles architecture detection automatically.

## Preparing Your ESRGAN Model

Before loading a custom model, you must place it in a directory indexed by ComfyUI's path system.

### Default Model Directory

By default, ComfyUI looks for upscaling models in `models/upscale_models/`. This path is defined in [`folder_paths.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/folder_paths.py):

```python
folder_names_and_paths["upscale_models"] = ([os.path.join(models_dir, "upscale_models")], supported_pt_extensions)

```

Copy or symlink your ESRGAN `.safetensors`, `.pth`, or `.ckpt` files into this directory:

```

ComfyUI/
└─ models/
   └─ upscale_models/
      ├─ ESRGAN_x4.safetensors
      └─ RealESRGAN_x4plus.safetensors

```

### External Model Paths Configuration

If you maintain models in a central repository outside the ComfyUI tree, expose the path via [`extra_model_paths.yaml`](https://github.com/Comfy-Org/ComfyUI/blob/main/extra_model_paths.yaml). Rename the provided `extra_model_paths.yaml.example` and add your directories:

```yaml

# extra_model_paths.yaml

upscale_models:
  - /path/to/central/ESRGAN
  - /path/to/RealESRGAN

```

Save the file in the ComfyUI root and restart the application so the new paths are added to `folder_paths["upscale_models"]`.

## Loading and Executing the Model

Once your checkpoint is accessible, use the built-in nodes to process images.

### The Load Upscale Model Node

In the ComfyUI editor:

1. Add a **Load Upscale Model** node from the *loaders* category
2. Select your checkpoint from the `model_name` dropdown (populated by `folder_paths.get_filename_list("upscale_models")`)
3. The node outputs an **Upscale Model** object containing the `spandrel.ImageModelDescriptor` and the model's native scale factor

The node executes this logic from [`comfy_extras/nodes_upscale_model.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy_extras/nodes_upscale_model.py) (lines 35-44):

```python
model_path = folder_paths.get_full_path_or_raise("upscale_models", model_name)
sd = comfy.utils.load_torch_file(model_path, safe_load=True)
out = ModelLoader().load_from_state_dict(sd).eval()
if not isinstance(out, ImageModelDescriptor):
    raise Exception("Upscale model must be a single-image model.")
return (out,)

```

### The Upscale Image (with Model) Node

To process images:

1. Add an **Upscale Image (with Model)** node from the `image/upscaling` category
2. Connect the **Upscale Model** output to the node's *upscale_model* input
3. Connect an `IMAGE` tensor to the *image* input
4. Execute the workflow

### Automatic Tiling and Memory Management

The `ImageUpscaleWithModel` node handles technical complexities automatically:

- **VRAM-aware allocation**: Calculates required memory and frees VRAM using `model_management.free_memory` before inference
- **Tiling strategy**: Splits images into 512px tiles with 32px overlap to prevent OOM errors
- **OOM fallback**: Automatically reduces tile size if CUDA runs out of memory
- **Scale respect**: Uses `upscale_model.scale` to determine output dimensions (typically 4× for ESRGAN)

The tiling logic (lines 78-92 in [`nodes_upscale_model.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/nodes_upscale_model.py)) ensures you can run large ESRGAN models even on GPUs with limited VRAM.

## Programmatic Implementation

For scripted workflows, invoke the nodes directly in Python:

```python
import torch
from comfy import folder_paths, model_management, utils
from comfy_extras.nodes_upscale_model import UpscaleModelLoader, ImageUpscaleWithModel

# 1. Load the model

model_name = "ESRGAN_x4.safetensors"
model_path = folder_paths.get_full_path_or_raise("upscale_models", model_name)
state = utils.load_torch_file(model_path, safe_load=True)
upscale_model = UpscaleModelLoader().execute(model_name)[0]

# 2. Load an image (as torch tensor, shape [1, H, W, 3])

img = utils.load_image("input.png")

# 3. Upscale

out = ImageUpscaleWithModel().execute(upscale_model, img)[0]
utils.save_image(out, "upscaled.png")

```

This script mirrors the UI implementation exactly, including automatic tiling and VRAM management.

## Troubleshooting Custom Models

| Issue | Solution |
|-------|----------|
| **Model not recognized** (error "Upscale model must be a single-image model") | Ensure the checkpoint contains a single-image `spandrel` descriptor. For custom architectures, create a wrapper class inheriting from `spandrel.ImageModelDescriptor` and register it via `spandrel_extra_arches` following the pattern in [`nodes_upscale_model.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/nodes_upscale_model.py). |
| **Different upscaling factor** (e.g., 2×, 3×) | Check `upscale_model.scale` to verify the model's native factor. The node respects this value automatically, but you can query it programmatically if you need to adjust downstream processing. |
| **Want to change tile size** | The default is 512px with 32px overlap. To adjust globally, modify `tile = 512` and `overlap = 32` in `ImageUpscaleWithModel.execute` within [`comfy_extras/nodes_upscale_model.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy_extras/nodes_upscale_model.py). |
| **Batch upscaling** | Feed a batch of images with shape `[N, H, W, 3]` into the node. The tiling logic processes each image in the batch automatically. |

## Summary

- **Place** your ESRGAN checkpoint in `models/upscale_models/` or expose it via [`extra_model_paths.yaml`](https://github.com/Comfy-Org/ComfyUI/blob/main/extra_model_paths.yaml) and restart ComfyUI.
- **Load** the model using the **Load Upscale Model** node, which wraps the checkpoint in a `spandrel.ImageModelDescriptor` via [`comfy_extras/nodes_upscale_model.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy_extras/nodes_upscale_model.py).
- **Process** images through the **Upscale Image (with Model)** node, which automatically handles 512px tiling, 32px overlap, VRAM management, and OOM recovery.
- **Script** the workflow programmatically using `UpscaleModelLoader` and `ImageUpscaleWithModel` classes for automated pipelines.

## Frequently Asked Questions

### Where does ComfyUI look for ESRGAN model files?

ComfyUI searches the `upscale_models` folder defined in [`folder_paths.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/folder_paths.py), which defaults to `ComfyUI/models/upscale_models/`. You can add additional search paths by creating an [`extra_model_paths.yaml`](https://github.com/Comfy-Org/ComfyUI/blob/main/extra_model_paths.yaml) file in the ComfyUI root and specifying `upscale_models` entries pointing to external directories.

### Why does my custom ESRGAN model throw "Upscale model must be a single-image model"?

This error occurs in [`comfy_extras/nodes_upscale_model.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy_extras/nodes_upscale_model.py) when the loaded checkpoint does not produce a `spandrel.ImageModelDescriptor` instance. Ensure your model follows the single-image super-resolution format. For custom architectures, you must create a wrapper class inheriting from `ImageModelDescriptor` and register it via the spandrel extra architectures system.

### How does ComfyUI handle large images that don't fit in GPU memory?

The `ImageUpscaleWithModel` node implements automatic tiling with a default size of 512px and 32px overlap. If CUDA runs out of memory, the node catches the OOM exception and retries with smaller tiles. This logic in [`comfy_extras/nodes_upscale_model.py`](https://github.com/Comfy-Org/ComfyUI/blob/main/comfy_extras/nodes_upscale_model.py) allows ESRGAN models to run on GPUs with as little as 1GB VRAM.

### Can I use ESRGAN models with different scale factors like 2× or 3×?

Yes. The `upscale_model.scale` attribute stores the model's native upscaling factor, which the `ImageUpscaleWithModel` node respects automatically. When loading a model, the `UpscaleModelLoader` extracts this value from the checkpoint metadata. You can verify the scale factor by checking the model output or querying `upscale_model.scale` in a custom script.