# How to Implement Background Removal with U-2-Net or MODNet in ailia-models

> Easily implement background removal using U-2-Net or MODNet with the ailia SDK. Learn how to process images efficiently with pre-trained ONNX models for stunning results.

- Repository: [axinc-ai/ailia-models](https://github.com/axinc-ai/ailia-models)
- Tags: how-to-guide
- Published: 2026-02-25

---

**Implement background removal in ailia-models by using the pre-trained U-2-Net or MODNet ONNX models with the ailia SDK, processing images through resize-normalization-inference pipelines available in [`background_removal/u2net/u2net.py`](https://github.com/axinc-ai/ailia-models/blob/main/background_removal/u2net/u2net.py) and [`background_removal/modnet/modnet.py`](https://github.com/axinc-ai/ailia-models/blob/main/background_removal/modnet/modnet.py).**

The **axinc-ai/ailia-models** repository provides production-ready implementations for **background removal** using deep learning. Both U-2-Net and MODNet are available as ONNX models wrapped by the ailia SDK, enabling high-performance alpha matting for images and video without requiring manual trimap creation.

## Understanding the Background Removal Architecture

The repository offers two distinct approaches to **background removal**: **U-2-Net** for salient object detection and **MODNet** for trimap-free portrait matting. Both models follow a consistent inference pattern defined in [`background_removal/u2net/u2net.py`](https://github.com/axinc-ai/ailia-models/blob/main/background_removal/u2net/u2net.py) and [`background_removal/modnet/modnet.py`](https://github.com/axinc-ai/ailia-models/blob/main/background_removal/modnet/modnet.py): automatic model download via `model_utils.check_and_download_models`, initialization with `ailia.Net`, pre-processing, inference with `net.predict`, and post-processing to generate the alpha matte.

## Implementing Background Removal with U-2-Net

### Model Architecture and Pre-processing

U-2-Net utilizes a nested U-structure encoder-decoder that aggregates multi-scale features through deep supervision. According to the source code in [`background_removal/u2net/u2net.py`](https://github.com/axinc-ai/ailia-models/blob/main/background_removal/u2net/u2net.py), the model accepts a default input size of 320×320 pixels and outputs a single-channel probability map (α mask). The script supports both the full-size model (`u2net.onnx`) and the lightweight variant (`u2netp.onnx`), selected via the model list configuration (lines 30-40).

### U-2-Net Implementation Example

The following implementation mirrors the CLI logic found in [`u2net.py`](https://github.com/axinc-ai/ailia-models/blob/main/u2net.py):

```python

# u2net_demo.py

import ailia
from image_utils import imread
from u2net_utils import load_image, norm, save_result, transform
from model_utils import check_and_download_models

# ---- Configuration -------------------------------------------------

WEIGHT = "u2net.onnx"                # or u2netp.onnx for the small model

PROTO  = "u2net.onnx.prototxt"
REMOTE = "https://storage.googleapis.com/ailia-models/u2net/"

# ---- Model preparation ----------------------------------------------

check_and_download_models(WEIGHT, PROTO, REMOTE)
net = ailia.Net(PROTO, WEIGHT, env_id=0)

# ---- Inference -------------------------------------------------------

input_path  = "input.jpg"
output_path = "output.png"

# 1. Load & resize to network size (default 320×320)

img, h, w = load_image(input_path, scaled_size=(320, 320), rgb_mode=False)

# 2. Forward pass

pred = net.predict([img])[0][0, 0, :, :]          # shape H×W

# 3. Restore original resolution & save

save_result(pred, output_path, [h, w])
print("Result saved to:", output_path)

```

Key implementation details from the source:

- The `load_image` function in [`u2net_utils.py`](https://github.com/axinc-ai/ailia-models/blob/main/u2net_utils.py) handles resizing and normalization
- Inference returns a tensor of shape 1×1×H×W, requiring extraction of the first channel
- The `--composite` flag (lines 44-48) adds an alpha channel to the original BGR image for direct compositing

## Implementing Background Removal with MODNet

### Model Architecture and Scaling Requirements

MODNet implements a lightweight encoder-decoder with three branches (detail, semantic, and fusion) that predicts a high-resolution matte without requiring a trimap. As implemented in [`background_removal/modnet/modnet.py`](https://github.com/axinc-ai/ailia-models/blob/main/background_removal/modnet/modnet.py), the model uses a reference height of 512 pixels (`INFERENCE_HEIGHT = 512`, line 41) and requires input dimensions to be multiples of 32. The `modnet_utils.get_scale_factor` function (lines 3-22) calculates the scaling factor to ensure this alignment.

### MODNet Implementation Example

This code reproduces the workflow from [`modnet.py`](https://github.com/axinc-ai/ailia-models/blob/main/modnet.py):

```python

# modnet_demo.py

import ailia
import cv2, numpy as np
from modnet_utils import get_scale_factor
from model_utils import check_and_download_models
from image_utils import imread

# ---- Configuration -------------------------------------------------

WEIGHT = "modnet.opt.onnx"
PROTO  = "modnet.opt.onnx.prototxt"
REMOTE = "https://storage.googleapis.com/ailia-models/modnet/"

# ---- Model preparation ----------------------------------------------

check_and_download_models(WEIGHT, PROTO, REMOTE)
net = ailia.Net(PROTO, WEIGHT, env_id=0)

# ---- Inference -------------------------------------------------------

img_path = "input.jpg"
out_path = "output.png"

raw = imread(img_path)               # BGR

rgb = cv2.cvtColor(raw, cv2.COLOR_BGR2RGB).astype("float32")
rgb = (rgb - 127.5) / 127.5          # normalise to [-1, 1]

h, w, _ = rgb.shape
sx, sy = get_scale_factor(h, w, 512) # reference height (line 41)

# Resize while keeping factor multiples of 32

rgb_resized = cv2.resize(rgb, None, fx=sx, fy=sy, interpolation=cv2.INTER_AREA)

# Prepare NCHW tensor

blob = np.transpose(rgb_resized, (2, 0, 1))[np.newaxis, ...].astype("float32")

# Forward

pred = net.predict(blob)[0]
matte = (np.squeeze(pred) * 255).astype("uint8")
matte = cv2.resize(matte, (w, h), interpolation=cv2.INTER_AREA)

# Optional compositing

rgba = cv2.cvtColor(raw, cv2.COLOR_BGR2BGRA)
rgba[:, :, 3] = matte
cv2.imwrite(out_path, rgba)
print("Result saved to:", out_path)

```

Critical differences from U-2-Net:

- **Normalization**: MODNet requires scaling to [-1, 1] rather than standard ImageNet normalization
- **Soft matte**: The output is a float 0-1 mask scaled to 0-255 (line 79), providing smoother edges than U-2-Net's binary-like output
- **Dimension constraints**: Width and height must be divisible by 32, enforced by `get_scale_factor`

## Comparing U-2-Net and MODNet for Background Removal

Both scripts expose a consistent CLI (`--input`, `--savepath`, `--video`, `--composite`) using the generic `arg_utils` helper, but differ in their computer vision approach:

- **U-2-Net**: Optimized for general salient object detection with a nested U-structure; best for arbitrary objects against varied backgrounds
- **MODNet**: Specialized for portrait matting with trimap-free inference; ideal for human subjects requiring fine hair detail preservation

## Summary

- **Background removal** in ailia-models is implemented via ONNX models wrapped by the `ailia.Net` class
- **U-2-Net** ([`background_removal/u2net/u2net.py`](https://github.com/axinc-ai/ailia-models/blob/main/background_removal/u2net/u2net.py)) processes 320×320 inputs using deep supervision for binary mask generation
- **MODNet** ([`background_removal/modnet/modnet.py`](https://github.com/axinc-ai/ailia-models/blob/main/background_removal/modnet/modnet.py)) processes 512×736 inputs with 32-pixel alignment constraints for soft alpha matting
- Both pipelines use `model_utils.check_and_download_models` for automatic weight management and support video processing via `recognize_from_video`

## Frequently Asked Questions

### What is the difference between U-2-Net and MODNet for background removal?

U-2-Net performs salient object detection outputting a binary mask suitable for general objects, while MODNet performs trimap-free portrait matting producing a soft alpha matte optimized for human subjects. U-2-Net uses a nested U-structure with 320×320 inputs, whereas MODNet employs a three-branch architecture requiring 32-pixel aligned inputs at 512×736 resolution.

### How do I process video files for background removal?

Both [`u2net.py`](https://github.com/axinc-ai/ailia-models/blob/main/u2net.py) and [`modnet.py`](https://github.com/axinc-ai/ailia-models/blob/main/modnet.py) implement `recognize_from_video` functions that capture frames using OpenCV, apply the same pre-processing pipeline as images, and optionally composite the matte onto the original frame in real-time. Use the `--video` CLI argument to specify the input video path.

### Can I run these models on CPU-only systems?

Yes. The `ailia.Net` initialization accepts an `env_id` parameter where `0` typically selects CPU inference. The ONNX models are optimized for the ailia SDK's CPU backend, though GPU acceleration is available when CUDA or Metal drivers are present.

### Where does the code download the model weights?

The `check_and_download_models` function in [`util/model_utils.py`](https://github.com/axinc-ai/ailia-models/blob/main/util/model_utils.py) automatically retrieves ONNX weights and prototxt files from Google Cloud Storage URLs (e.g., `https://storage.googleapis.com/ailia-models/u2net/`) if they are not present in the local working directory.