How to Optimize RIFE Inference Speed for 4K Video Using the `--scale` Parameter

Use the --scale parameter (or the --UHD flag) to downsample 4K frames before optical flow estimation, reducing GPU memory usage and computation time by processing fewer pixels at each pyramid level of the RIFE network.

RIFE (Real-Time Intermediate Flow Estimation) interpolates intermediate frames by estimating optical flow between consecutive frames. When processing 4K video (3840 × 2160), the computational cost grows quadratically with resolution, often exceeding GPU memory limits or real-time processing constraints. According to the hzwer/eccv2022-rife source code, the --scale parameter provides a direct mechanism to trade resolution for speed by controlling the internal multi-scale pyramid resolution.

Understanding the --scale Parameter in RIFE

The RIFE network uses a multi-scale pyramid architecture to compute optical flow. At full 4K resolution (scale=1.0), the network processes the entire pixel grid across all pyramid levels, which requires substantial VRAM and compute cycles. The --scale parameter reduces the effective resolution fed into the network while maintaining the architectural constraints required for valid optical flow estimation.

How --scale Works Under the Hood

The scale factor operates at three critical points in the inference pipeline:

1. Padding Calculation for Network Alignment

Before inference, RIFE calculates padding to ensure tensor dimensions respect the network's 32-pixel alignment requirements. In inference_video.py lines 198-202, the padding block size tmp is computed inversely to the scale factor:

tmp = max(32, int(32 / scale))

When scale=0.5, the padding increases to 64 pixels, ensuring the downsampled tensors remain valid for the network's convolutional layers. This adjustment happens in the preprocessing stage before frames enter the core model.

2. Multi-Scale Pyramid Rescaling

Inside model/RIFE.py lines 56-61, the scale parameter rescales the internal scale_list that drives the pyramid network:

scale_list = [8, 4, 2, 1]
for i in range(4):
    scale_list[i] = scale_list[i] * 1.0 / scale

With the default scale=0.5 (UHD mode), this transforms the pyramid to [16, 8, 4, 2]—effectively operating on coarser feature maps that require fewer computations per level. This rescaling allows the backbone to compute optical flow at reduced resolution while maintaining the hierarchical refinement structure.

3. Forward Pass Resolution

During the forward pass in model/RIFE.py lines 62-66, the concatenated frames are processed by self.flownet using the rescaled scale_list. Because the number of pixels processed at each pyramid level is reduced proportionally to the square of the scale factor, runtime drops dramatically. A scale of 0.5 processes one-quarter the pixels of full resolution, while 0.25 processes one-sixteenth.

The --UHD Convenience Flag

For 4K workflows, the repository provides the --UHD flag in inference_video.py lines 74-76, which automatically sets scale=0.5 when the input resolution exceeds HD thresholds:

if args.UHD:
    scale = 0.5

This provides a sensible default for 4K content without requiring manual scale calculations.

Practical Implementation: Command-Line Optimization

Recommended Settings for 4K Video

For optimal 4K performance using the built-in UHD shortcut, combine the automatic scaling with half-precision inference:

python inference_video.py \
    --video input_4k.mp4 \
    --output output_4k.mp4 \
    --UHD \
    --fp16 \
    --exp 2

The --UHD flag triggers the 0.5 scale factor, while --fp16 enables Tensor Core acceleration on modern NVIDIA GPUs (handled in inference_video.py lines 85-87). The --exp 2 parameter generates 3 intermediate frames (2² - 1) between each source frame.

Aggressive Speed Optimization

For maximum frame rate on limited hardware, manually specify a more aggressive downsample factor:

python inference_video.py \
    --video input_4k.mp4 \
    --output output_fast.mp4 \
    --scale 0.25 \
    --fp16 \
    --exp 1

Setting --scale 0.25 reduces the effective resolution by 75%, cutting pixel processing to one-sixteenth of full 4K. This configuration suits preview generation or real-time applications where speed outweighs quality requirements.

Programmatic Control Using the Python API

When integrating RIFE into custom video processing pipelines, pass the scale parameter directly to the Model.inference() method:

import torch
import cv2
from model.RIFE import Model

# Initialize model (HD version)

model = Model()
model.load_model('train_log', rank=-1)
model.eval()
model.device()

# Load 4K frames (BGR to RGB conversion)

frame0 = cv2.imread('frame0.png')[:, :, ::-1] / 255.0
frame1 = cv2.imread('frame1.png')[:, :, ::-1] / 255.0

# Convert to CUDA tensors

I0 = torch.from_numpy(frame0.transpose(2, 0, 1)).unsqueeze(0).float().cuda()
I1 = torch.from_numpy(frame1.transpose(2, 0, 1)).unsqueeze(0).float().cuda()

# Inference with 0.5 scale for 4K optimization

mid = model.inference(I0, I1, scale=0.5)
mid_np = (mid.squeeze(0).cpu().numpy().transpose(1, 2, 0) * 255).astype('uint8')
cv2.imwrite('interpolated.png', mid_np[:, :, ::-1])

The scale=0.5 argument triggers the same pyramid rescaling ([8, 4, 2] effective levels) used by the --UHD CLI flag, as implemented in model/RIFE.py.

Additional Performance Tips

Beyond the --scale parameter, optimize 4K RIFE inference with these strategies:

• Enable Half-Precision (--fp16) – In inference_video.py lines 85-87, this flag converts the model to FP16, halving memory bandwidth and accelerating convolution operations on GPUs with Tensor Cores.
• Avoid PNG Output – Omit the --png flag unless you specifically need frame-wise PNG sequences. Writing directly to video containers (MP4/MKV) reduces I/O overhead significantly.
• Manage Interpolation Exponent (--exp) – The exponent determines how many intermediate frames are generated (2^exp - 1). While RIFE processes frames sequentially with a fixed batch size of 1, higher exp values multiply total compute time. Use --exp 1 for 2x frame rate or --exp 2 for 4x, balancing smoothness against processing time.

Summary

• The --scale parameter in RIFE controls the resolution of the multi-scale pyramid, directly impacting GPU memory and compute requirements for 4K video.
• Setting --scale 0.5 (or using --UHD) quarters the pixel count processed by the network, enabling real-time or near-real-time 4K interpolation on modern GPUs.
• The parameter rescales the internal scale_list in model/RIFE.py and adjusts padding calculations in inference_video.py to maintain valid network alignment.
• Combine --scale 0.5 with --fp16 for optimal speed/quality balance, or use --scale 0.25 for maximum performance when quality can be compromised.
• Programmatic users can pass scale=0.5 directly to model.inference(I0, I1, scale=0.5) for custom pipeline integration.

Frequently Asked Questions

What is the default scale value when using the --UHD flag?

The --UHD flag automatically sets the scale to 0.5 for high-resolution inputs. In inference_video.py lines 74-76, the code explicitly assigns scale = 0.5 when the UHD argument is present, providing a 50% resolution reduction optimized for 4K content without requiring manual calculation.

Does lowering the scale parameter affect video quality?

Yes, reducing the scale factor decreases the resolution at which optical flow is estimated, which can reduce motion boundary precision and fine detail preservation. However, for many 4K video sequences (particularly those with smooth motion), the visual degradation at scale=0.5 remains minimal while providing substantial performance gains. Values below 0.5 (such as 0.25) introduce more noticeable artifacts but enable processing on hardware that cannot otherwise handle 4K inputs.

Can I use the scale parameter with custom Python scripts?

Absolutely. When using the Python API from model/RIFE.py, pass the scale argument directly to the inference() method: model.inference(img0, img1, scale=0.5). This programmatic approach provides the same pyramid rescaling behavior as the command-line interface, allowing integration into custom video processing pipelines or real-time applications.

What GPU memory requirements should I expect for 4K RIFE interpolation?

At full 4K resolution (scale=1.0), RIFE requires approximately 10-12 GB of VRAM depending on the specific model variant. Using --scale 0.5 typically reduces this to 6-8 GB, while --scale 0.25 can bring requirements below 4 GB. For reference, an RTX 3080 (10 GB) can comfortably process 4K video with scale=0.5 and --fp16 enabled, whereas full-resolution 4K processing requires high-end cards like the RTX 3090 or A6000.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}