How to Configure DFRPipeline Temporal Upsampling Rounds in LTX-2

Set temporal_upsample_rounds to 0, 1, or 2 in the __call__ method, and provide a valid temporal_upsampler_path when rounds > 0.

The DFRPipeline in Lightricks' LTX-2 repository supports optional temporal upsampling that doubles frame rates after spatial rendering. This guide explains how to configure the temporal_upsample_rounds parameter and its dependencies using the actual source implementation.

DFRPipeline Temporal Upsampling Overview

The pipeline implements a two-stage diffusion workflow: spatial rendering followed by optional temporal refinement. Temporal upsampling occurs in a dedicated loop that processes latents through a VideoUpsampler to achieve smoother motion.

Key constraints from packages/ltx-pipelines/src/ltx_pipelines/dfr_pipeline.py:

• temporal_upsample_rounds must be 0, 1, or 2 (enforced at lines 77-78)
• temporal_upsampler_path is required when rounds > 0 (enforced at lines 79-82)

Each round doubles the frame count and playback FPS (30 → 60 → 120 fps). The underlying mechanism uses tiled diffusion processing to maintain temporal consistency across expanded frame sequences.

Configuration Parameters

temporal_upsample_rounds Argument

Pass this directly to the pipeline's __call__ method:

Value	Effect	Output FPS (from 30 fps input)
0	No upsampling	30 fps
1	One 2× upsampling round	60 fps
2	Two 2× upsampling rounds	120 fps

temporal_upsampler_path Requirement

When temporal_upsample_rounds > 0, you must specify a path to a compatible x2 latent upsampler model. The pipeline validates this at initialization and raises ValueError if missing.

In dfr_pipeline.py lines 35-46, the pipeline creates self.temporal_upsampler as a VideoUpsampler instance wrapping the loaded model.

Complete Configuration Examples

No Temporal Upsampling (Default)

Use this for fastest generation when motion smoothness is not critical:

from ltx_pipelines import DFRPipeline

pipeline = DFRPipeline(
    model_paths=paths,
    distilled_lora=distilled,
    spatial_upsampler_path="spatial_up.pt",
    loras=user_loras,
    temporal_upsampler_path=None,  # Not required when rounds=0

)

video, audio, n_frames, cfg = pipeline(
    prompt="A sunrise over a city",
    seed=42,
    height=720,
    width=1280,
    frame_rate=30.0,
    images=[],
    temporal_upsample_rounds=0,  # Disables temporal upsampling

)

Single Round: 2× FPS (60 fps output)

One upsampling round doubles frame rate and refines motion:

pipeline = DFRPipeline(
    model_paths=paths,
    distilled_lora=distilled,
    spatial_upsampler_path="spatial_up.pt",
    loras=user_loras,
    temporal_upsampler_path="temporal_up_x2.pt",  # Required for upsampling

)

video, audio, n_frames, cfg = pipeline(
    prompt="A sunrise over a city",
    seed=42,
    height=720,
    width=1280,
    frame_rate=30.0,
    images=[],
    temporal_upsample_rounds=1,  # One x2 round → 60 fps

)

Two Rounds: 4× FPS (120 fps output)

Maximum temporal quality for slow-motion effects:

pipeline = DFRPipeline(
    model_paths=paths,
    distilled_lora=distilled,
    spatial_upsampler_path="spatial_up.pt",
    loras=user_loras,
    temporal_upsampler_path="temporal_up_x2.pt",  # Same model reused per round

)

video, audio, n_frames, cfg = pipeline(
    prompt="A sunrise over a city",
    seed=42,
    height=720,
    width=1280,
    frame_rate=30.0,
    images=[],
    temporal_upsample_rounds=2,  # Two x2 rounds → 120 fps

)

How Temporal Upsampling Works Internally

The implementation in dfr_pipeline.py (lines 302-342) processes each round through four stages:

1. Latent upsampling: self.temporal_upsampler(video_state.latent[:1]) applies the x2 upsampler
2. Frame doubling: num_frames = 2 * (num_frames - 1) + 1 expands temporal dimension
3. Tiled diffusion: Canvas splits into 2**round_idx temporal tiles processed separately
4. Stitching: stitch_tile_latents recombines per-tile results

Key-frame consistency across rounds is maintained by _merge_carry_forward_keyframes (lines 27-40 of dfr_pipeline.py), which propagates anchor frames through the upsampling sequence.

The tiling logic resides in dfr_layout.py, providing tile_ranges for temporal segmentation and stitch_tile_latents for reconstruction.

Source File Reference

File	Purpose	Key Components
packages/ltx-pipelines/src/ltx_pipelines/dfr_pipeline.py	Core pipeline implementation	DFRPipeline.__call__, temporal upsampling loop, parameter validation
packages/ltx-pipelines/src/ltx_pipelines/dfr_layout.py	Tiling and stitching utilities	tile_ranges, stitch_tile_latents
packages/ltx-pipelines/src/ltx_pipelines/utils/blocks.py	Model wrappers	VideoUpsampler class definition

Summary

• Valid values: temporal_upsample_rounds accepts only 0, 1, or 2
• Model requirement: temporal_upsampler_path is mandatory when rounds > 0
• Performance trade-off: Each round doubles output FPS but increases generation time through additional diffusion steps
• Implementation location: Core logic in dfr_pipeline.py lines 77-82 (validation) and 302-342 (execution loop)

Frequently Asked Questions

What happens if I set temporal_upsample_rounds=2 without providing temporal_upsampler_path?

The pipeline raises a ValueError during initialization. The check at lines 79-82 of dfr_pipeline.py validates that self.temporal_upsampler is not None when rounds exceed zero.

Can I use different upsampler models for each round?

No. The same VideoUpsampler instance is reused across all rounds. The temporal_upsampler_path loads a single x2 latent upsampler that doubles resolution repeatedly—two rounds apply the same model twice sequentially.

Why does frame count follow 2 * (num_frames - 1) + 1 instead of simple doubling?

This formula accounts for overlapping frame windows in the latent representation. The overlapping structure preserves temporal continuity at boundaries when tiles are processed independently during each upsampling round.

How does temporal_upsample_rounds affect generation time?

Each round adds a full diffusion pass over progressively larger frame counts. One round roughly doubles temporal compute; two rounds roughly quadruple it, though tiled processing in dfr_layout.py provides some parallelism mitigation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}