# How to Configure FP8 Quantization in LTX-2: `fp8-cast` vs `fp8-scaled-mm` for Different GPU Architectures

> Configure FP8 quantization in LTX-2 using fp8-cast or fp8-scaled-mm. Optimize performance for different GPU architectures with the --quantization flag or policy objects.

- Repository: [Lightricks/LTX-2](https://github.com/Lightricks/LTX-2)
- Tags: deep-dive
- Published: 2026-08-18

---

**LTX-2 offers two FP8 quantization policies—`fp8-cast` for universal compatibility and `fp8-scaled-mm` for maximum performance on Hopper-class GPUs—configured via the `--quantization` CLI flag or programmatic policy objects.**

FP8 quantization in LTX-2 reduces memory footprint by approximately 50% while maintaining generation quality. The framework adapts to your hardware through two distinct implementation strategies: one that prioritizes broad GPU support and another that leverages native FP8 matrix-multiply instructions. Understanding these differences ensures you select the optimal configuration for your NVIDIA GPU architecture.

---

## FP8 Quantization Policies: At a Glance

LTX-2 implements FP8 through two mutually exclusive policies defined in the `ltx_core.quantization` package:

| Policy | Storage Format | Inference Math | GPU Requirement | Best For |
|--------|---------------|----------------|-----------------|----------|
| **`fp8-cast`** | Weights cast to FP8 at load time | Up-cast to BF16/FP16, then standard `matmul` | Any GPU supporting FP8 storage (Ampere+) | Consumer GPUs, maximum compatibility |
| **`fp8-scaled-mm`** | Pre-quantized FP8 weights + scale tensors | Native `torch._scaled_mm` in FP8 | Hopper SM90+ (native FP8 units) | Data center GPUs, maximum throughput |

The CLI flag `--quantization` accepts these as `fp8-cast` or `fp8-scaled-mm`, parsed by `QuantizationKind` in [`ltx_pipelines/utils/quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/ltx_pipelines/utils/quantization_factory.py).

---

## `fp8-cast`: Universal Compatibility

The **`fp8-cast`** policy stores transformer linear weights in FP8 after loading, then up-casts them on-the-fly during inference. This approach requires no special hardware acceleration.

### How It Works

In [`packages/ltx-core/src/ltx_core/quantization/fp8_cast.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_cast.py), the policy builds two core components:

1. **Load-time transformation** (`_build_prequant_fold_sd_ops`): Converts BF16 checkpoint weights to `float8_e4m3fn` using the `TRANSFORMER_LINEAR_DOWNCAST_MAP`
2. **Runtime up-casting** (`_upcast_and_round`): Converts FP8 weights back to the input dtype (typically BF16) inside each linear layer

```python

# From fp8_cast.py: the fuse rule for LoRA merging

def fp8_cast_fuse_rule(up_weight: torch.Tensor, down_weight: torch.Tensor) -> torch.Tensor:
    # Merged weight remains in FP8 storage dtype

    return (up_weight @ down_weight).to(torch.float8_e4m3fn)

```

### Hardware Support

**Any GPU with FP8 tensor storage support** can use `fp8-cast`. This includes:

- NVIDIA Ampere (SM 80-86): RTX 30-series, A10, A30
- NVIDIA Ada Lovelace (SM 89): RTX 40-series, A6000 Ada
- NVIDIA Hopper (SM 90): H100, H200
- NVIDIA Blackwell (SM 100+): B100, B200

The actual matrix multiplication executes in BF16 or FP16, so performance matches non-quantized inference with ~2× memory savings.

---

## `fp8-scaled-mm`: Native FP8 Acceleration

The **`fp8-scaled-mm`** policy utilizes PyTorch's `torch._scaled_mm` to perform matrix multiplication directly in FP8, avoiding the up-cast overhead entirely.

### How It Works

In [`packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py), the `FP8Linear` class implements:

1. **Pre-quantized weights**: Checkpoint contains `.weight` (FP8) and `.weight_scale` (per-tensor float)
2. **Fused dequantization + matmul**: `torch._scaled_mm(weight, input, scale_a=weight_scale, scale_b=input_scale)`
3. **Scale folding**: The fuse rule (`fp8_scaled_mm_fuse_rule`) merges LoRA updates while preserving scale tensors

```python

# From fp8_scaled_mm.py: FP8Linear forward pass

def forward(self, x: torch.Tensor) -> torch.Tensor:
    # x is quantized to FP8 with dynamic scale

    x_fp8, x_scale = self._quantize_input(x)
    # Native FP8 matmul with fused scaling

    return torch._scaled_mm(
        self.weight,           # FP8 weight

        x_fp8.T,               # FP8 input

        scale_a=self.weight_scale,
        scale_b=x_scale,
        out_dtype=torch.bfloat16,
    )

```

### Hardware Requirements

**Requires Hopper SM90+ for full acceleration.** The `torch._scaled_mm` operation:

- Uses **dedicated FP8 Tensor Cores** on H100/H200/B100/B200
- Falls back to emulation on older GPUs, eliminating performance benefits
- Provides up to **1.5× speedup** over `fp8-cast` at large batch sizes

Check your GPU capability programmatically:

```python
import torch

major, minor = torch.cuda.get_device_capability()
if major >= 9:  # Hopper or newer

    print("Native FP8 acceleration available")
else:
    print("Use fp8-cast for compatibility")

```

---

## CLI Configuration

Select your quantization policy via the `--quantization` flag in any LTX-2 pipeline:

```bash

# fp8-cast: works on any FP8-capable GPU (recommended default)

python -m ltx_pipelines.ti2vid_two_stages \
    --quantization fp8-cast \
    --checkpoint-path ./ltx-2.3-bf16.safetensors \
    --prompt "A cinematic drone shot over mountains at sunset" \
    --output-path output.mp4

# fp8-scaled-mm: requires pre-quantized checkpoint + Hopper GPU

python -m ltx_pipelines.ti2vid_two_stages \
    --quantization fp8-scaled-mm \
    --checkpoint-path ./ltx-2.3-fp8.safetensors \
    --prompt "A cinematic drone shot over mountains at sunset" \
    --output-path output_fast.mp4

```

The flag parsing resides in [`packages/ltx-pipelines/src/ltx_pipelines/utils/args.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/utils/args.py) lines 99-104, with policy construction delegated to `QuantizationKind` (lines 22-42 in [`quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/quantization_factory.py)).

---

## Programmatic Configuration

For custom integrations, build policies directly:

```python
from ltx_core.quantization.fp8_cast import build_policy as build_fp8_cast
from ltx_core.quantization.fp8_scaled_mm import build_policy as build_fp8_scaled_mm
from ltx_pipelines.ti2vid_two_stages import TI2VidTwoStagesPipeline
from ltx_pipelines.utils.model_paths import ModelPaths

# Automatic hardware-based selection

import torch

def get_optimal_policy(checkpoint_path: str):
    major, _ = torch.cuda.get_device_capability()
    if major >= 9:
        # Hopper+: use native FP8 matmul

        return build_fp8_scaled_mm(checkpoint_path)
    # Older GPUs: safe cast-based approach

    return build_fp8_cast(checkpoint_path)

# Initialize pipeline with selected policy

policy = get_optimal_policy("./ltx-2.3.safetensors")
pipeline = TI2VidTwoStagesPipeline(
    model_paths=ModelPaths.from_monolith(
        transformer_path="./ltx-2.3.safetensors",
        gemma_root_path="./gemma-2b-it",
    ),
    quantization=policy,
    # ... other parameters

)

```

The `QuantizationPolicy` object (from `ltx_core.quantization.policy`) encapsulates:
- **SD operations**: Checkpoint transformation rules
- **Fuse rules**: LoRA merging behavior for each quantization type

---

## GPU Architecture Recommendations

| GPU Architecture | Compute Capability | Recommended Policy | Expected Memory Savings | Speed Impact |
|------------------|-------------------:|-------------------:|------------------------:|-------------:|
| Ampere (A10, A30, RTX 3090) | SM 86 | `fp8-cast` | ~50% | Neutral |
| Ada Lovelace (RTX 4090, A6000 Ada) | SM 89 | `fp8-cast` | ~50% | Neutral |
| Hopper (H100, H200) | SM 90 | `fp8-scaled-mm` | ~50% | **~1.5× faster** |
| Blackwell (B100, B200) | SM 100+ | `fp8-scaled-mm` | ~50% | **~1.5× faster** |

Use `fp8-cast` as the default unless you have confirmed Hopper/Blackwell hardware and a pre-quantized FP8 checkpoint.

---

## Checkpoint Compatibility

- **`fp8-cast`**: Accepts standard BF16 checkpoints (e.g., `ltx-2.3-bf16.safetensors`). Down-casting occurs at load time.
- **`fp8-scaled-mm`**: Requires **pre-quantized checkpoints** containing both `weight` (FP8) and `weight_scale` tensors (e.g., `ltx-2.3-fp8.safetensors`). Attempting to use BF16 checkpoints with `fp8-scaled-mm` will raise a runtime error.

---

## Summary

- **Use `fp8-cast`** for maximum compatibility across all FP8-capable GPUs; it stores weights in FP8 but computes in BF16
- **Use `fp8-scaled-mm`** exclusively on Hopper (H100) or newer GPUs with pre-quantized checkpoints for native FP8 acceleration
- Configure via `--quantization fp8-cast|fp8-scaled-mm` CLI flag or `build_policy()` functions programmatically
- Check GPU capability with `torch.cuda.get_device_capability()` to select the optimal policy automatically
- Reference implementation files: [`fp8_cast.py`](https://github.com/Lightricks/LTX-2/blob/main/fp8_cast.py), [`fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/fp8_scaled_mm.py), and [`quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/quantization_factory.py)

---

## Frequently Asked Questions

### What happens if I use `fp8-scaled-mm` on an older GPU without native FP8 support?

PyTorch falls back to a software emulation path for `torch._scaled_mm`, which eliminates the performance advantage and may run slower than `fp8-cast`. The operation will still produce correct results, but you lose the ~1.5× speedup that makes `fp8-scaled-mm` worthwhile. Use `torch.cuda.get_device_capability()` to verify SM ≥ 90 before selecting this policy.

### Can I convert a BF16 checkpoint to work with `fp8-scaled-mm`?

Not through LTX-2's built-in tools. The `fp8-scaled-mm` policy expects pre-quantized checkpoints with per-tensor scale factors already computed. Converting BF16 to this format requires calibrated quantization-aware training or post-training quantization with scale estimation. For standard checkpoints, use `fp8-cast` which performs the conversion automatically at load time.

### Does FP8 quantization affect video generation quality?

Both policies maintain perceptual quality comparable to BF16 inference in LTX-2. The `fp8-cast` path uses round-trip quantization (BF16 → FP8 → BF16), while `fp8-scaled-mm` uses the same FP8 format throughout with proper scaling. No significant quality degradation has been observed in the LTX-2 evaluation suite.

### How do I verify which quantization policy is active at runtime?

Inspect the `pipeline.quantization` attribute after construction. The policy object reveals its type through the underlying SD operations and fuse rules. For CLI runs, add `--verbose` logging to emit the selected policy name during initialization.