# How to Integrate RIFE with VapourSynth for AI Video Frame Interpolation

> Integrate RIFE with VapourSynth for AI video frame interpolation. Learn how to load models and use stdModifyFrame or stdInterleave for advanced video processing pipelines.

- Repository: [hzwer/eccv2022-rife](https://github.com/hzwer/eccv2022-rife)
- Tags: how-to-guide
- Published: 2026-03-03

---

**You can integrate RIFE with VapourSynth by loading the pretrained `Model` class from [`model/RIFE.py`](https://github.com/hzwer/eccv2022-rife/blob/main/model/RIFE.py), wrapping the `inference()` method to handle tensor conversion, and using `std.ModifyFrame` or `std.Interleave` to insert interpolated frames into your processing pipeline.**

The hzwer/eccv2022-rife repository provides a high-performance PyTorch implementation of RIFE (Real-Time Intermediate Flow Estimation) for synthesizing intermediate frames between video frames. While the repository ships with a command-line driver, integrating RIFE directly into VapourSynth pipelines requires bridging the PyTorch model with VS's frame-based processing graph.

## Understanding the RIFE Architecture

### Core Model Components

The interpolation logic resides in [`model/RIFE.py`](https://github.com/hzwer/eccv2022-rife/blob/main/model/RIFE.py), which defines the **`Model`** class. This class encapsulates the IFNet architecture for optical flow estimation and the fusion network for frame synthesis. The two primary methods for integration are:

- `load_model(path, rank)`: Loads pretrained weights from `flownet.pkl` and associated checkpoint files in the specified directory.
- `inference(im0, im1, scale)`: Accepts two PyTorch tensors of shape `(1, 3, H, W)` with values in `[0, 1]` and returns the interpolated middle frame.

### Reference Implementation

The [`inference_video.py`](https://github.com/hzwer/eccv2022-rife/blob/main/inference_video.py) script demonstrates practical usage of the `Model` class. It handles video I/O, iterates through frame pairs, and contains the `make_inference()` helper function (lines 78-88) for recursive interpolation when generating multiple intermediate frames for 4× or 8× slow-motion.

## Step-by-Step VapourSynth Integration

### Step 1: Load the Pretrained Model

First, instantiate the RIFE model and load the pretrained weights. The model directory should contain `flownet.pkl` and related checkpoint files downloaded from the repository releases.

```python
from model.RIFE import Model
import torch
from pathlib import Path

model_dir = Path('train_log')  # Directory containing flownet.pkl

rife = Model()
rife.load_model(str(model_dir), rank=0)
rife.eval()
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
rife.flownet.to(device)

```

### Step 2: Wrap Model Inference for VS Frames

VapourSynth frames use the `VideoFrame` object with plane-based storage. Create a helper function that converts VS frames to PyTorch tensors, calls `rife.inference()`, and returns the result as a numpy array.

```python
import vapoursynth as vs
import numpy as np

core = vs.core

def interpolate_pair(frame0: vs.VideoFrame, frame1: vs.VideoFrame) -> np.ndarray:
    """Generate intermediate frame between two VS frames."""
    # Convert VS planes to HxWxC numpy array

    img0 = np.stack([np.array(frame0.get_read_array(i), copy=False) 
                     for i in range(frame0.format.num_planes)], axis=2)
    img1 = np.stack([np.array(frame1.get_read_array(i), copy=False) 
                     for i in range(frame1.format.num_planes)], axis=2)
    
    # Convert to torch tensors (1,3,H,W) in [0,1]

    t0 = torch.from_numpy(img0).permute(2, 0, 1).unsqueeze(0).float().to(device) / 255.0
    t1 = torch.from_numpy(img1).permute(2, 0, 1).unsqueeze(0).float().to(device) / 255.0
    
    with torch.no_grad():
        middle = rife.inference(t0, t1, scale=1)
    
    # Convert back to numpy HxWx3 uint8

    out = (middle[0].cpu().numpy().transpose(1, 2, 0) * 255.0).clip(0, 255).astype(np.uint8)
    return out

```

### Step 3: Inject Interpolated Frames into the Pipeline

Use VapourSynth's `std.ModifyFrame` or `std.Interleave` to insert the generated frames between original frames. For 2× interpolation, alternate between original and interpolated frames.

```python
def rife_double_fps(clip: vs.VideoNode) -> vs.VideoNode:
    """Double the frame rate using RIFE interpolation."""
    # Cache frames to allow look-ahead

    frame_list = [clip.get_frame(i) for i in range(clip.num_frames)]
    new_frames = []
    
    for i in range(len(frame_list) - 1):
        new_frames.append(frame_list[i])
        mid_np = interpolate_pair(frame_list[i], frame_list[i+1])
        # Create VS frame from numpy (simplified - in production use proper frame creation)

        mid_frame = core.std.BlankClip(width=clip.width, height=clip.height, 
                                       length=1, format=vs.RGB24)
        new_frames.append(mid_frame)
    
    new_frames.append(frame_list[-1])
    
    # Rebuild clip

    def get_frame(n):
        return new_frames[n]
    
    return core.std.ModifyFrame(
        template=core.std.BlankClip(width=clip.width, height=clip.height, 
                                    length=len(new_frames), fpsnum=clip.fps_num*2, 
                                    fpsden=clip.fps_den, format=clip.format),
        clip=clip,
        selector=get_frame
    )

# Usage

src = core.ffms2.Source('input.mp4')
src = core.resize.Bicubic(src, format=vs.RGB24)  # RIFE requires RGB

out = rife_double_fps(src)
out.set_output()

```

## Achieving Higher Interpolation Factors

For 4× or 8× slow-motion, implement recursive interpolation using the `make_inference` pattern from [`inference_video.py`](https://github.com/hzwer/eccv2022-rife/blob/main/inference_video.py). This function generates intermediate frames between already-interpolated results.

```python
def make_inference(I0, I1, n):
    """Recursively generate n intermediate frames between I0 and I1."""
    if n == 0:
        return []
    middle = rife.inference(I0, I1, scale=1)
    if n == 1:
        return [middle]
    # Recursively interpolate between I0-middle and middle-I1

    left = make_inference(I0, middle, n // 2)
    right = make_inference(middle, I1, n // 2)
    return left + [middle] + right

```

Set `n` to `2^exp - 1` where `exp` is the desired exponent (2 for 4×, 3 for 8×).

## Alternative: Community VapourSynth Plugins

If you prefer not to maintain custom glue code, community plugins provide ready-made VS integration:

- **[vs-rife](https://github.com/HolyWu/vs-rife)**: A Python-based plugin that wraps the official implementation. Install via `pip install vsrife` and invoke as `core.rife.RIFE(clip, model='4.6', num_frames=2)`.

- **[VapourSynth-RIFE-ncnn-Vulkan](https://github.com/styler00dollar/VapourSynth-RIFE-ncnn-Vulkan)**: A Vulkan-accelerated port using ncnn for GPU inference without PyTorch dependencies, ideal for systems lacking CUDA support.

## Summary

- The hzwer/eccv2022-rife repository provides the **`Model`** class in [`model/RIFE.py`](https://github.com/hzwer/eccv2022-rife/blob/main/model/RIFE.py) for frame interpolation, with `load_model()` and `inference()` as the primary integration points.
- To integrate with VapourSynth, convert VS `VideoFrame` objects to PyTorch tensors, run `rife.inference()`, and convert the output back to numpy arrays for frame reconstruction.
- Use `std.ModifyFrame` or `std.Interleave` to inject interpolated frames into the pipeline, effectively doubling or quadrupling frame rates.
- For higher-order interpolation (4×, 8×), implement recursive inference using the `make_inference` helper pattern from [`inference_video.py`](https://github.com/hzwer/eccv2022-rife/blob/main/inference_video.py).
- Community plugins like **vs-rife** and **VapourSynth-RIFE-ncnn-Vulkan** offer drop-in alternatives if you prefer not to write custom Python glue code.

## Frequently Asked Questions

### Can I use RIFE with VapourSynth without installing PyTorch?

While the official hzwer/eccv2022-rife implementation requires PyTorch for the `Model` class, you can use the **VapourSynth-RIFE-ncnn-Vulkan** community plugin instead. This port uses the ncnn inference engine and Vulkan compute shaders, eliminating the PyTorch and CUDA dependencies while maintaining real-time performance on compatible GPUs.

### What input format does RIFE expect when integrated into VapourSynth?

RIFE expects **RGB24** planar format with pixel values normalized to the range `[0, 1]` as PyTorch tensors of shape `(1, 3, H, W)`. In your VapourSynth script, convert your source clip using `core.resize.Bicubic(src, format=vs.RGB24)` before processing. The `inference()` method in [`model/RIFE.py`](https://github.com/hzwer/eccv2022-rife/blob/main/model/RIFE.py) handles the tensor operations internally.

### How do I handle memory management for long videos in VapourSynth?

For long videos, avoid caching all frames in a Python list simultaneously as shown in the basic example. Instead, implement a sliding window buffer using `std.FrameEval` to process frame pairs on-demand, or use the **vs-rife** plugin which handles frame caching internally. When using the raw Python approach, ensure you wrap inference calls in `torch.no_grad()` and manually delete intermediate tensors (`del tensor_name`) to prevent GPU memory accumulation.

### Where can I find pretrained model weights for the integration?

Pretrained weights are available in the [releases section](https://github.com/hzwer/eccv2022-rife/releases) of the hzwer/eccv2022-rife repository. Download the `flownet.pkl` and associated checkpoint files, then place them in a directory such as `train_log/`. In your integration script, point the `load_model()` method to this directory. The [`inference_video.py`](https://github.com/hzwer/eccv2022-rife/blob/main/inference_video.py) reference implementation demonstrates this pattern by defaulting to the `train_log` directory for checkpoint loading.