How to Implement Super-Resolution for Image Enhancement with AILIA-Models

Implement super-resolution for image enhancement by downloading pre-trained ONNX weights with check_and_download_models, initializing ailia.Net, preprocessing images to NCHW format with normalization, running net.predict, and post-processing the output tensor back to HWC BGR format for saving.

The axinc-ai/ailia-models repository provides production-ready implementations of state-of-the-art super-resolution models for image enhancement. Whether you need to upscale photos, restore compressed JPEGs, or enhance anime artwork, these scripts offer a unified pipeline built on the AILIA inference engine.

Standard Super-Resolution Pipeline Architecture

Every super-resolution script in the repository follows a consistent six-step architecture. Understanding this flow allows you to adapt any model for custom image enhancement tasks.

1. Argument Parsing and CLI Setup

Scripts use shared utilities from util/arg_utils.py to build consistent command-line interfaces. The get_base_parser function creates a foundation with standard flags for input paths, output paths, and benchmarking.

from util.arg_utils import get_base_parser, update_parser, get_savepath

parser = get_base_parser('Single Image Super-Resolution', 'input.png', 'output.png')
args = update_parser(parser)

2. Model Download and Verification

The check_and_download_models function in util/model_utils.py automatically fetches ONNX weights and prototxt files from Google Cloud Storage if they are not present locally.

from util.model_utils import check_and_download_models

WEIGHT_PATH = 'srresnet.opt.onnx'
MODEL_PATH = 'srresnet.opt.onnx.prototxt'
REMOTE_PATH = 'https://storage.googleapis.com/ailia-models/srresnet/'

check_and_download_models(WEIGHT_PATH, MODEL_PATH, REMOTE_PATH)

3. Network Initialization

Models are loaded using ailia.Net with optional memory optimization flags. For SwinIR, ONNX Runtime is also supported as an alternative backend.

import ailia

# Memory-efficient mode for large images

memory_mode = ailia.get_memory_mode(
    reduce_constant=True,
    ignore_input_with_initializer=True,
    reduce_interstage=False,
    reuse_interstage=True
)

net = ailia.Net(MODEL_PATH, WEIGHT_PATH, env_id=args.env_id, memory_mode=memory_mode)

4. Pre-processing

Input images are loaded using utilities from util/image_utils.py, normalized to the expected range (0-255 for most AILIA models, 0-1 for SwinIR), and reshaped from HWC to NCHW format.

from util.image_utils import load_image
import numpy as np

# Load and normalize

input_data = load_image(
    args.input[0],
    (64, 64),  # LR patch size

    normalize_type='255',
    gen_input_ailia=True
)

# Set input shape for dynamic batching

net.set_input_shape((1, 3, 64, 64))

5. Inference

The predict method runs the super-resolution transformation, returning a high-resolution tensor with dimensions scaled by the model's upscale factor (typically 2× or 4×).

# Run super-resolution

sr_tensor = net.predict(input_data)[0]  # Shape: (3, 256, 256) for 4x upscaling

6. Post-processing and Saving

The output tensor is transposed back to HWC format, converted from RGB to BGR for OpenCV compatibility, clamped to valid pixel ranges, and saved to disk.

import cv2
from util.arg_utils import get_savepath

# Convert to HWC and BGR

sr_image = sr_tensor.transpose(1, 2, 0)
sr_image = cv2.cvtColor(sr_image, cv2.COLOR_RGB2BGR)

# Save result

save_path = get_savepath(args.savepath, args.input[0])
cv2.imwrite(save_path, sr_image * 255)  # Scale back to 0-255

print(f'Super-resolved image saved to {save_path}')

Available Super-Resolution Models

The repository provides multiple model families optimized for different image enhancement scenarios. Each model is implemented in its own subdirectory under super_resolution/.

SRResNet

SRResNet provides fast, high-quality 4× upscaling with optional tiling support for very large images. The implementation in super_resolution/srresnet/srresnet.py offers both optimized and standard ONNX variants.

python super_resolution/srresnet/srresnet.py \
    -i input.jpg \
    -o output.png \
    --padding  # Enable tiling for large images

Real-ESRGAN

Real-ESRGAN delivers state-of-the-art perceptual quality for both photographs and anime-style images. The script in super_resolution/real-esrgan/real_esrgan.py handles alpha channels and offers specialized models for different content types.

python super_resolution/real-esrgan/real_esrgan.py \
    -i input.png \
    -o output.png \
    -m RealESRGAN_anime  # Use anime-optimized model

SwinIR

SwinIR leverages Swin Transformer architecture for classical SR, lightweight SR, real-world SR, and JPEG artifact removal. Implemented in super_resolution/swinir/swinir.py, it supports both AILIA and ONNX Runtime backends.

python super_resolution/swinir/swinir.py \
    -i compressed.jpg \
    --model_name jpeg  # JPEG denoising mode

Additional Models

• EDSR (super_resolution/edsr/edsr.py): Enhanced Deep Super-Resolution with bilinear fallback for video processing
• RCAN-IT (super_resolution/rcan-it/rcan-it.py): Image Transformer variant of RCAN with memory-efficient tiling
• HAN (super_resolution/han/han.py): Hierarchical Attention Network for single-pass inference
• HAT (super_resolution/hat/hat.py): Hybrid Attention Transformer with lightweight implementation
• SPAN (super_resolution/span/span.py): Spatial Attention Network optimized for fast inference

Utility Modules for Custom Implementation

When building custom super-resolution pipelines, leverage the shared utility modules to maintain consistency with the repository's architecture.

Argument Utilities (util/arg_utils.py)

Provides get_base_parser, update_parser, and get_savepath for standardized CLI interfaces across all super-resolution scripts.

Model Utilities (util/model_utils.py)

The check_and_download_models function handles automatic downloading of ONNX weights from remote storage, verifying file integrity before inference.

Image Utilities (util/image_utils.py)

Contains load_image, imread, and get_image_shape for reading and normalizing images into the format expected by AILIA networks.

WebCamera Utilities (util/webcamera_utils.py)

Provides video capture, frame preprocessing, and writer creation utilities for processing video streams through super-resolution models.

Summary

• Super-resolution for image enhancement in ailia-models follows a standardized six-step pipeline: CLI parsing, model download, network initialization, preprocessing, inference, and post-processing.
• The repository provides nine distinct model families including SRResNet, Real-ESRGAN, and SwinIR, each optimized for specific use cases from fast 4× upscaling to JPEG artifact removal.
• Shared utility modules in util/ provide consistent argument parsing, automatic model downloading, and image preprocessing across all super-resolution implementations.
• All scripts support both image and video processing, with optional tiling mechanisms for handling high-resolution inputs on memory-constrained devices.

Frequently Asked Questions

What is the difference between SRResNet and Real-ESRGAN for image enhancement?

SRResNet provides fast, deterministic 4× upscaling optimized for speed, while Real-ESRGAN focuses on perceptual quality with specialized variants for photographs and anime-style images. Real-ESRGAN in super_resolution/real-esrgan/real_esrgan.py also handles alpha channels, making it suitable for PNG images with transparency, whereas SRResNet in super_resolution/srresnet/srresnet.py offers optional tiling for very large images.

How do I handle memory constraints when processing large images with super-resolution models?

Use the tiling or padding options available in most scripts. For example, SRResNet supports --padding to process large images in overlapping tiles, while RCAN-IT in super_resolution/rcan-it/rcan-it.py includes built-in tiling logic for memory-efficient inference. Additionally, initialize ailia.Net with memory_mode parameters set to reduce_constant=True and reuse_interstage=True to minimize GPU memory usage.

Can I use these super-resolution models for video enhancement?

Yes, most super-resolution scripts in the repository support video processing through the -v flag. The scripts utilize util/webcamera_utils.py for frame capture and writing. For video-specific optimization, the EDSR implementation in super_resolution/edsr/edsr.py includes an optional bilinear fallback mode to maintain temporal consistency across frames when full super-resolution processing is too computationally expensive.

What preprocessing steps are required before running inference on images?

Images must be normalized to the range expected by the specific model—typically 0-255 for AILIA models or 0-1 for SwinIR—and reshaped from HWC (Height-Width-Channels) to NCHW (Batch-Channels-Height-Width) format. Use util/image_utils.py functions like load_image with normalize_type='255' and gen_input_ailia=True to handle these transformations automatically. Additionally, set the input shape using net.set_input_shape((1, 3, height, width)) before calling net.predict.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}