# How RIFE Implements Arbitrary-Timestep Video Frame Interpolation Using Recursive Inference

> Discover how RIFE achieves arbitrary-timestep video frame interpolation via recursive inference. Learn its innovative approach to generating intermediate frames dynamically.

- Repository: [hzwer/eccv2022-rife](https://github.com/hzwer/eccv2022-rife)
- Tags: deep-dive
- Published: 2026-03-03

---

**RIFE achieves arbitrary-timestep interpolation by conditioning its flow-estimation network on a continuous time-offset input and recursively bisecting the interval between two frames to generate any number of intermediate frames.**

The RIFE (Realtime Intermediate Flow Estimation) framework, available in the `hzwer/eccv2022-rife` repository, enables high-quality video frame interpolation at any temporal position between two input frames. Unlike fixed midpoint interpolation, **arbitrary-timestep video frame interpolation** requires the model to predict motion and appearance for continuous time values. RIFE solves this through a time-aware network architecture combined with a recursive inference strategy that optimizes both quality and computational efficiency.

## Time-Offset Conditioning in IFNet_m

When initialized with `arbitrary=True`, the RIFE model loads the **IFNet_m** architecture from [`model/IFNet_m.py`](https://github.com/hzwer/eccv2022-rife/blob/main/model/IFNet_m.py) instead of the standard IFNet. This variant accepts a continuous **timestep** parameter that conditions the entire flow estimation process.

### Continuous Time Encoding

The network receives the normalized time offset `t ∈ [0, 1]` as an additional channel broadcast to the full spatial dimensions. In the `forward` method of `IFNet_m`, the timestep tensor is concatenated with the input frames:

```python

# IFNet_m.forward (lines 64-66)

timestep = (x[:, :1].clone() * 0 + 1) * timestep      # broadcast scalar to full-size map

img0 = x[:, :3]
img1 = x[:, 3:6]
...

# First block receives the time map together with the two input frames

flow, mask = stu[i](torch.cat((img0, img1, timestep), 1), None, scale=scale[i])

```

This conditioning allows the network to learn a **continuous mapping from (I₀, I₁, t) → (optical flow, blending mask)**, enabling direct prediction for any intermediate moment rather than just the midpoint.

The high-level `Model` class in [`model/RIFE.py`](https://github.com/hzwer/eccv2022-rife/blob/main/model/RIFE.py) forwards this timestep parameter during inference:

```python

# Model.inference (model/RIFE.py, lines 56-66)

flow, mask, merged, flow_teacher, merged_teacher, loss_distill = \
    self.flownet(imgs, scale_list, timestep=timestep)

```

When `timestep` is omitted, the network defaults to `0.5` for standard midpoint interpolation. Any other value produces an arbitrary-timestep frame directly.

## Recursive Bisection for Multiple Arbitrary Frames

For generating multiple intermediate frames (e.g., 2ⁿ–1 frames for 2ⁿ-fold slow-motion), RIFE employs a **recursive bisection strategy** rather than invoking the network separately for each timestamp. This approach, implemented in [`inference_video.py`](https://github.com/hzwer/eccv2022-rife/blob/main/inference_video.py), minimizes forward passes while maintaining temporal coherence.

### The Recursive Inference Strategy

The `make_inference` function recursively subdivides the temporal interval:

```python

# make_inference (inference_video.py, lines 78-88)

def make_inference(I0, I1, n):
    middle = model.inference(I0, I1, args.scale)   # default t=0.5

    if n == 1:
        return [middle]
    first_half  = make_inference(I0, middle, n=n//2)
    second_half = make_inference(middle, I1, n=n//2)
    if n % 2:
        return [*first_half, middle, *second_half]
    else:
        return [*first_half, *second_half]

```

The algorithm follows these steps:

1. **Base case**: Generate the midpoint frame using `t = 0.5`.
2. **Recursive step**: Split the interval `[I₀, I₁]` into two sub-intervals and repeat the process on each half.
3. **Composition**: Merge the results from both halves, inserting the middle frame when necessary.

Because each recursive call leverages the time-offset-conditioned network, the final sequence corresponds to **exactly the requested arbitrary timestamps** (e.g., ¼, ¾, ⅛). This method requires only **O(log₂ N)** forward passes for N frames, significantly reducing computational overhead compared to direct per-frame inference.

## End-to-End Implementation Flow

The complete arbitrary-timestep pipeline involves three distinct stages:

1. **Model initialization**: `model = Model(arbitrary=True)` selects `IFNet_m` and enables time conditioning.
2. **Single-frame inference**: `model.inference(I0, I1, timestep=0.3)` returns the frame at 30% of the temporal interval.
3. **Multi-frame generation**: `make_inference(I0, I1, n)` recursively constructs sequences of arbitrary length.

This architecture ensures that RIFE can handle both single arbitrary-timestep queries and complex multi-frame interpolation tasks efficiently.

## Complete Working Example

The following implementation demonstrates loading the arbitrary-timestep model and generating both single and multiple intermediate frames:

```python

# -------------------------------------------------

# 1. Load the arbitrary-timestep model

# -------------------------------------------------

from model.RIFE import Model
model = Model(arbitrary=True)        # internally selects IFNet_m

model.load_model('train_log', -1)
model.eval()
model.device()

# -------------------------------------------------

# 2. Interpolate a single arbitrary frame (t = 0.2)

# -------------------------------------------------

I0 = torch.randn(1, 3, 720, 1280).to(device)   # first frame tensor

I1 = torch.randn(1, 3, 720, 1280).to(device)   # second frame tensor

mid = model.inference(I0, I1, timestep=0.2)      # frame at 20% of the interval

# mid is a tensor of shape (1, 3, H, W)

# -------------------------------------------------

# 3. Generate 7 intermediate frames (2³-fold slow-motion)

# -------------------------------------------------

def make_inference(I0, I1, n):
    middle = model.inference(I0, I1)            # default t=0.5

    if n == 1:
        return [middle]
    first = make_inference(I0, middle, n // 2)
    second = make_inference(middle, I1, n // 2)
    return [*first, middle, *second] if n % 2 else [*first, *second]

frames = make_inference(I0, I1, n=7)   # returns a list of 7 tensors

```

## Summary

- **Time-offset conditioning**: The `IFNet_m` architecture in [`model/IFNet_m.py`](https://github.com/hzwer/eccv2022-rife/blob/main/model/IFNet_m.py) accepts a continuous `timestep` parameter that broadcasts to a full spatial map, enabling direct prediction of optical flow and blending masks at any temporal position.
- **Recursive bisection**: The `make_inference` function in [`inference_video.py`](https://github.com/hzwer/eccv2022-rife/blob/main/inference_video.py) recursively halves temporal intervals, generating multiple arbitrary-timestep frames with **O(log₂ N)** forward passes rather than linear complexity.
- **Flexible API**: Setting `arbitrary=True` during model initialization switches from standard midpoint interpolation to the time-aware architecture, supporting both single-frame queries (`timestep=0.3`) and complex sequence generation.

## Frequently Asked Questions

### What is the difference between IFNet and IFNet_m in RIFE?

The standard **IFNet** architecture performs fixed midpoint interpolation between two frames. **IFNet_m**, located in [`model/IFNet_m.py`](https://github.com/hzwer/eccv2022-rife/blob/main/model/IFNet_m.py), extends this capability by accepting a continuous `timestep` parameter that conditions the flow estimation process. When `arbitrary=True` is passed to the `Model` constructor, RIFE automatically loads `IFNet_m` to enable arbitrary-timestep interpolation.

### How does the timestep parameter affect interpolation quality?

The `timestep` parameter represents the normalized temporal position between two input frames, where `0.0` corresponds to the first frame and `1.0` to the second. The network learns to predict optical flow and blending masks conditioned on this continuous value, allowing it to generate physically plausible intermediate frames at any sub-frame precision. The quality depends on the network's ability to generalize across the continuous time domain during training.

### Why does RIFE use recursive bisection instead of direct inference for multiple frames?

Recursive bisection reduces the computational complexity from **O(N)** to **O(log₂ N)** forward passes for generating N intermediate frames. By recursively interpolating the midpoint and then processing sub-intervals, RIFE reuses intermediate results and maintains temporal consistency across the sequence. This approach is implemented in [`inference_video.py`](https://github.com/hzwer/eccv2022-rife/blob/main/inference_video.py) and is particularly efficient for generating 2ⁿ–1 intermediate frames (e.g., 7 frames for 8× slow-motion).

### What is the computational complexity of generating N arbitrary frames?

Using the recursive strategy in `make_inference`, generating N intermediate frames requires approximately **⌈log₂(N+1)⌉** forward passes through the network. For example, producing 7 intermediate frames (8× slow-motion) requires only 3 recursive levels (midpoint, then quarters, then eighths), totaling 7 forward passes rather than 7 separate inferences. This logarithmic scaling makes high-frame-rate interpolation computationally feasible.