How to Configure FP8 Quantization in LTX-2: `fp8-cast` vs `fp8-scaled-mm` for Different GPU Architectures

LTX-2 offers two FP8 quantization policies—fp8-cast for universal compatibility and fp8-scaled-mm for maximum performance on Hopper-class GPUs—configured via the --quantization CLI flag or programmatic policy objects.

FP8 quantization in LTX-2 reduces memory footprint by approximately 50% while maintaining generation quality. The framework adapts to your hardware through two distinct implementation strategies: one that prioritizes broad GPU support and another that leverages native FP8 matrix-multiply instructions. Understanding these differences ensures you select the optimal configuration for your NVIDIA GPU architecture.

---

FP8 Quantization Policies: At a Glance

LTX-2 implements FP8 through two mutually exclusive policies defined in the ltx_core.quantization package:

Policy	Storage Format	Inference Math	GPU Requirement	Best For
fp8-cast	Weights cast to FP8 at load time	Up-cast to BF16/FP16, then standard matmul	Any GPU supporting FP8 storage (Ampere+)	Consumer GPUs, maximum compatibility
fp8-scaled-mm	Pre-quantized FP8 weights + scale tensors	Native torch._scaled_mm in FP8	Hopper SM90+ (native FP8 units)	Data center GPUs, maximum throughput

The CLI flag --quantization accepts these as fp8-cast or fp8-scaled-mm, parsed by QuantizationKind in ltx_pipelines/utils/quantization_factory.py.

---

fp8-cast: Universal Compatibility

The fp8-cast policy stores transformer linear weights in FP8 after loading, then up-casts them on-the-fly during inference. This approach requires no special hardware acceleration.

How It Works

In packages/ltx-core/src/ltx_core/quantization/fp8_cast.py, the policy builds two core components:

1. Load-time transformation (_build_prequant_fold_sd_ops): Converts BF16 checkpoint weights to float8_e4m3fn using the TRANSFORMER_LINEAR_DOWNCAST_MAP
2. Runtime up-casting (_upcast_and_round): Converts FP8 weights back to the input dtype (typically BF16) inside each linear layer

# From fp8_cast.py: the fuse rule for LoRA merging

def fp8_cast_fuse_rule(up_weight: torch.Tensor, down_weight: torch.Tensor) -> torch.Tensor:
    # Merged weight remains in FP8 storage dtype

    return (up_weight @ down_weight).to(torch.float8_e4m3fn)

Hardware Support

Any GPU with FP8 tensor storage support can use fp8-cast. This includes:

• NVIDIA Ampere (SM 80-86): RTX 30-series, A10, A30
• NVIDIA Ada Lovelace (SM 89): RTX 40-series, A6000 Ada
• NVIDIA Hopper (SM 90): H100, H200
• NVIDIA Blackwell (SM 100+): B100, B200

The actual matrix multiplication executes in BF16 or FP16, so performance matches non-quantized inference with ~2× memory savings.

---

fp8-scaled-mm: Native FP8 Acceleration

The fp8-scaled-mm policy utilizes PyTorch's torch._scaled_mm to perform matrix multiplication directly in FP8, avoiding the up-cast overhead entirely.

How It Works

In packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py, the FP8Linear class implements:

1. Pre-quantized weights: Checkpoint contains .weight (FP8) and .weight_scale (per-tensor float)
2. Fused dequantization + matmul: torch._scaled_mm(weight, input, scale_a=weight_scale, scale_b=input_scale)
3. Scale folding: The fuse rule (fp8_scaled_mm_fuse_rule) merges LoRA updates while preserving scale tensors

# From fp8_scaled_mm.py: FP8Linear forward pass

def forward(self, x: torch.Tensor) -> torch.Tensor:
    # x is quantized to FP8 with dynamic scale

    x_fp8, x_scale = self._quantize_input(x)
    # Native FP8 matmul with fused scaling

    return torch._scaled_mm(
        self.weight,           # FP8 weight

        x_fp8.T,               # FP8 input

        scale_a=self.weight_scale,
        scale_b=x_scale,
        out_dtype=torch.bfloat16,
    )

Hardware Requirements

Requires Hopper SM90+ for full acceleration. The torch._scaled_mm operation:

• Uses dedicated FP8 Tensor Cores on H100/H200/B100/B200
• Falls back to emulation on older GPUs, eliminating performance benefits
• Provides up to 1.5× speedup over fp8-cast at large batch sizes

Check your GPU capability programmatically:

import torch

major, minor = torch.cuda.get_device_capability()
if major >= 9:  # Hopper or newer

    print("Native FP8 acceleration available")
else:
    print("Use fp8-cast for compatibility")

---

CLI Configuration

Select your quantization policy via the --quantization flag in any LTX-2 pipeline:

# fp8-cast: works on any FP8-capable GPU (recommended default)

python -m ltx_pipelines.ti2vid_two_stages \
    --quantization fp8-cast \
    --checkpoint-path ./ltx-2.3-bf16.safetensors \
    --prompt "A cinematic drone shot over mountains at sunset" \
    --output-path output.mp4

# fp8-scaled-mm: requires pre-quantized checkpoint + Hopper GPU

python -m ltx_pipelines.ti2vid_two_stages \
    --quantization fp8-scaled-mm \
    --checkpoint-path ./ltx-2.3-fp8.safetensors \
    --prompt "A cinematic drone shot over mountains at sunset" \
    --output-path output_fast.mp4

The flag parsing resides in packages/ltx-pipelines/src/ltx_pipelines/utils/args.py lines 99-104, with policy construction delegated to QuantizationKind (lines 22-42 in quantization_factory.py).

---

Programmatic Configuration

For custom integrations, build policies directly:

from ltx_core.quantization.fp8_cast import build_policy as build_fp8_cast
from ltx_core.quantization.fp8_scaled_mm import build_policy as build_fp8_scaled_mm
from ltx_pipelines.ti2vid_two_stages import TI2VidTwoStagesPipeline
from ltx_pipelines.utils.model_paths import ModelPaths

# Automatic hardware-based selection

import torch

def get_optimal_policy(checkpoint_path: str):
    major, _ = torch.cuda.get_device_capability()
    if major >= 9:
        # Hopper+: use native FP8 matmul

        return build_fp8_scaled_mm(checkpoint_path)
    # Older GPUs: safe cast-based approach

    return build_fp8_cast(checkpoint_path)

# Initialize pipeline with selected policy

policy = get_optimal_policy("./ltx-2.3.safetensors")
pipeline = TI2VidTwoStagesPipeline(
    model_paths=ModelPaths.from_monolith(
        transformer_path="./ltx-2.3.safetensors",
        gemma_root_path="./gemma-2b-it",
    ),
    quantization=policy,
    # ... other parameters

)

The QuantizationPolicy object (from ltx_core.quantization.policy) encapsulates:

• SD operations: Checkpoint transformation rules
• Fuse rules: LoRA merging behavior for each quantization type

---

GPU Architecture Recommendations

GPU Architecture	Compute Capability	Recommended Policy	Expected Memory Savings	Speed Impact
Ampere (A10, A30, RTX 3090)	SM 86	fp8-cast	~50%	Neutral
Ada Lovelace (RTX 4090, A6000 Ada)	SM 89	fp8-cast	~50%	Neutral
Hopper (H100, H200)	SM 90	fp8-scaled-mm	~50%	~1.5× faster
Blackwell (B100, B200)	SM 100+	fp8-scaled-mm	~50%	~1.5× faster

Use fp8-cast as the default unless you have confirmed Hopper/Blackwell hardware and a pre-quantized FP8 checkpoint.

---

Checkpoint Compatibility

• fp8-cast: Accepts standard BF16 checkpoints (e.g., ltx-2.3-bf16.safetensors). Down-casting occurs at load time.
• fp8-scaled-mm: Requires pre-quantized checkpoints containing both weight (FP8) and weight_scale tensors (e.g., ltx-2.3-fp8.safetensors). Attempting to use BF16 checkpoints with fp8-scaled-mm will raise a runtime error.

---

Summary

• Use fp8-cast for maximum compatibility across all FP8-capable GPUs; it stores weights in FP8 but computes in BF16
• Use fp8-scaled-mm exclusively on Hopper (H100) or newer GPUs with pre-quantized checkpoints for native FP8 acceleration
• Configure via --quantization fp8-cast|fp8-scaled-mm CLI flag or build_policy() functions programmatically
• Check GPU capability with torch.cuda.get_device_capability() to select the optimal policy automatically
• Reference implementation files: fp8_cast.py, fp8_scaled_mm.py, and quantization_factory.py

---

Frequently Asked Questions

What happens if I use fp8-scaled-mm on an older GPU without native FP8 support?

PyTorch falls back to a software emulation path for torch._scaled_mm, which eliminates the performance advantage and may run slower than fp8-cast. The operation will still produce correct results, but you lose the ~1.5× speedup that makes fp8-scaled-mm worthwhile. Use torch.cuda.get_device_capability() to verify SM ≥ 90 before selecting this policy.

Can I convert a BF16 checkpoint to work with fp8-scaled-mm?

Not through LTX-2's built-in tools. The fp8-scaled-mm policy expects pre-quantized checkpoints with per-tensor scale factors already computed. Converting BF16 to this format requires calibrated quantization-aware training or post-training quantization with scale estimation. For standard checkpoints, use fp8-cast which performs the conversion automatically at load time.

Does FP8 quantization affect video generation quality?

Both policies maintain perceptual quality comparable to BF16 inference in LTX-2. The fp8-cast path uses round-trip quantization (BF16 → FP8 → BF16), while fp8-scaled-mm uses the same FP8 format throughout with proper scaling. No significant quality degradation has been observed in the LTX-2 evaluation suite.

How do I verify which quantization policy is active at runtime?

Inspect the pipeline.quantization attribute after construction. The policy object reveals its type through the underlying SD operations and fuse rules. For CLI runs, add --verbose logging to emit the selected policy name during initialization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}