What Causes Disordered Patterns in RIFE Interpolation Output and How to Fix Them

Disordered patterns in RIFE interpolation output are caused by the optical flow network processing frames at resolutions too high for its receptive field, and you can eliminate these checkerboard-like artifacts by setting the --scale=2.0 flag to force the model to operate on a coarser internal grid.

The RIFE (Real-Time Intermediate Flow Estimation) framework provides efficient video frame interpolation, but users often encounter disordered patterns—checkerboard-like artifacts—in the output when processing high-resolution videos. Understanding what causes these visual distortions and how the resolution scaling mechanism works in the hzwer/eccv2022-rife repository is essential for producing clean interpolation results.

Understanding the Root Cause of Disordered Patterns in Interpolation Output

Disordered patterns emerge when the optical-flow network processes frames at a resolution that exceeds its effective receptive field or available GPU memory limits. In the RIFE architecture, the flow estimator IFNet operates on multi-scale versions of input frames to compute intermediate flows. When the internal processing resolution is too high relative to the model's capacity, the network produces noisy flow vectors that manifest as checkerboard artifacts in the final interpolated frames.

The repository's documentation explicitly identifies this relationship. According to the README in hzwer/eccv2022-rife:

"If you generate disordered pattern on your videos, try set --scale=2.0. This parameter controls the process resolution for the optical‑flow model."

How the Scale Parameter Controls Interpolation Resolution

The --scale parameter in inference_video.py directly modulates the internal resolution at which the optical flow network operates. The CLI definition restricts accepted values to [0.25, 0.5, 1.0, 2.0, 4.0] to ensure valid scaling factors:

# inference_video.py (lines 64-77)

parser.add_argument('--scale', dest='scale', type=float, default=1.0,
                    help='Try scale=0.5 for 4k video')

# ...

assert args.scale in [0.25, 0.5, 1.0, 2.0, 4.0]

Inside model/RIFE.py, the Model.inference method constructs a base scale list of [4, 2, 1] representing the multi-resolution pyramid levels. It then divides each entry by the user-provided scale factor:

# model/RIFE.py (lines 56-61)

def inference(self, img0, img1, scale=1, scale_list=None, TTA=False, timestep=0.5):
    if scale_list is None:
        scale_list = [4, 2, 1]
    for i in range(3):
        scale_list[i] = scale_list[i] * 1.0 / scale   # scale adjustment

When you set --scale=2.0, the internal scale list becomes [2, 1, 0.5], forcing the flow network to process features at half the spatial resolution. This coarser grid effectively increases the receptive field relative to pixel details, smoothing out the noisy flow vectors that cause disordered patterns.

Step-by-Step Solutions to Fix Disordered Patterns

Solution 1: Increase the Scale Factor for Problematic Videos

When checkerboard artifacts appear in your interpolated output, immediately increase the --scale parameter to 2.0. This is the primary fix recommended by the repository maintainers:

# Standard interpolation (may show artifacts on some videos)

python3 inference_video.py --exp=2 --video=video.mp4

# Fix for disordered patterns

python3 inference_video.py --exp=2 --video=video.mp4 --scale=2.0

Setting --scale=2.0 reduces the internal processing resolution by a factor of two, which eliminates the disordered patterns by ensuring the optical flow network operates within its effective receptive field.

Solution 2: Adjust Scale for High-Resolution Inputs

For 4K or higher resolution videos, the default scale may cause memory issues or artifacts. In these cases, down-scale the processing resolution by setting --scale=0.5:

# For 4K video to reduce memory usage and prevent artifacts

python3 inference_video.py --exp=2 --video=4k_video.mp4 --scale=0.5

This adjustment doubles the internal scale factors, allowing the model to process the high-resolution content more efficiently while maintaining flow accuracy.

Solution 3: Apply Fixes to Image Interpolation

The same disordered pattern issues can occur when interpolating between image pairs using inference_img.py. The --scale parameter works identically in this script:

# Normal image interpolation

python3 inference_img.py --img img0.png img1.png --exp=4

# With artifact fix

python3 inference_img.py --img img0.png img1.png --exp=4 --scale=2.0

The inference_img.py script forwards the --scale argument to the same Model.inference routine used by the video pipeline, ensuring consistent behavior across both interpolation modes.

Summary

• Disordered patterns (checkerboard artifacts) in RIFE interpolation output occur when the optical flow network processes frames at resolutions exceeding its receptive field capacity.
• The --scale parameter in inference_video.py and inference_img.py controls the internal processing resolution by modifying the multi-scale pyramid [4, 2, 1] used by the flow estimator in model/RIFE.py.
• Setting --scale=2.0 forces the network to operate on a coarser grid (effectively [2, 1, 0.5]), which eliminates disordered patterns by smoothing noisy flow vectors.
• For 4K videos, use --scale=0.5 to prevent memory issues while maintaining quality.
• The fix applies to both video (inference_video.py) and image (inference_img.py) interpolation workflows.

Frequently Asked Questions

What exactly are disordered patterns in RIFE interpolation?

Disordered patterns are checkerboard-like visual artifacts that appear in interpolated frames when the optical flow network generates noisy or inconsistent flow vectors. These artifacts manifest as grid-like distortions in the motion-compensated output, particularly visible in areas with complex motion or texture, and indicate that the network is operating outside its effective receptive field for the given input resolution.

Why does increasing --scale to 2.0 fix the artifacts?

Increasing --scale to 2.0 reduces the internal resolution at which the optical flow network processes features by dividing the default multi-scale pyramid [4, 2, 1] by the scale factor, resulting in [2, 1, 0.5]. This coarser grid effectively increases the network's receptive field relative to pixel details, allowing the flow estimator to capture larger motion patterns without generating the high-frequency noise that manifests as disordered checkerboard patterns.

Can I use any value for the --scale parameter?

No, the --scale parameter is restricted to specific discrete values defined in inference_video.py. The code explicitly asserts that args.scale must be one of [0.25, 0.5, 1.0, 2.0, 4.0]. Attempting to use values outside this set will trigger an assertion error, as these specific factors ensure the multi-scale pyramid divisions produce valid internal resolutions for the optical flow network.

Do I need to modify the source code to fix disordered patterns?

No source code modification is required to fix disordered patterns. The repository provides the --scale command-line flag specifically for this purpose, as documented in the README and implemented in inference_video.py and inference_img.py. Simply passing --scale=2.0 when running the inference scripts applies the necessary resolution adjustment internally through the Model.inference method in model/RIFE.py, eliminating the artifacts without changing any code.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}