# FP8 Quantization Backends in LTX-2: FP8-Cast and FP8-Scaled-MM Explained

> Explore FP8 quantization backends FP8-Cast and FP8-Scaled-MM in LTX-2. Discover which backend suits your checkpoint format and inference needs. Learn more now.

- Repository: [Lightricks/LTX-2](https://github.com/Lightricks/LTX-2)
- Tags: deep-dive
- Published: 2026-06-21

---

**LTX-2 supports two distinct FP8 quantization backends—FP8-Cast and FP8-Scaled-MM—exposed through the `QuantizationKind` enum in [`quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/quantization_factory.py), each optimized for different checkpoint formats and inference workflows.**

The LTX-2 video generation framework from Lightricks provides efficient low-precision inference through FP8 quantization. Understanding the supported FP8 quantization backends is essential for optimizing memory usage and computational performance when deploying or training models. Both backends implement the `QuantizationPolicy` interface but differ in how they store weights and handle scale tensors at runtime.

## Supported FP8 Quantization Backends

LTX-2 defines its FP8 quantization backends as entries in the `QuantizationKind` enum located in [`packages/ltx-pipelines/src/ltx_pipelines/utils/quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/utils/quantization_factory.py). Each backend returns a `QuantizationPolicy` via the `to_policy(checkpoint_path)` method, determining how model weights are loaded and how FP8 arithmetic executes.

### FP8-Cast Backend

The **FP8-Cast** backend (`fp8-cast`) stores linear weights in the FP8 (float8-e4m3) format and up-casts them to the input dtype during inference. This implementation folds any pre-quantized scale tensors (`*_scale`) into the weight matrix during model loading, eliminating the need for separate scale storage at runtime.

According to the LTX-2 source code in [`packages/ltx-core/src/ltx_core/quantization/fp8_cast.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_cast.py), this backend is ideal for BF16 checkpoints that contain pre-quantized scale tensors but require dynamic up-casting for compatibility with mixed-precision pipelines.

### FP8-Scaled-MM Backend

The **FP8-Scaled-MM** backend (`fp8-scaled-mm`) utilizes a scaled matrix-multiply kernel where FP8 weights remain paired with per-tensor `weight_scale` tensors. Unlike the cast backend, this approach performs de-quantization on-the-fly within the kernel, requiring checkpoints that already contain native FP8 weights and matching `.weight_scale` tensors.

This implementation lives in [`packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py) and is optimized for scenarios where the checkpoint has been fully quantized to FP8 with explicit scale metadata.

## Configuration and Usage

Both backends share a unified API through the `QuantizationKind` dispatcher. You can instantiate either policy programmatically or via the LTX-2 CLI.

### Python API Configuration

To select a backend in Python, import `QuantizationKind` and call `to_policy()` with your checkpoint path:

```python
from ltx_pipelines.utils.quantization_factory import QuantizationKind

checkpoint = "/path/to/my_fp8_checkpoint.safetensors"

# FP8-Cast: For BF16 checkpoints with scale tensors

policy_cast = QuantizationKind.FP8_CAST.to_policy(checkpoint)

# FP8-Scaled-MM: For native FP8 checkpoints with weight_scale tensors

policy_scaled = QuantizationKind.FP8_SCALED_MM.to_policy(checkpoint)

# Use with pipeline runners

# ltx_pipelines.ti2vid_one_stage.run(..., quantization_policy=policy_cast)

```

### CLI Configuration

The `ltx-trainer` CLI accepts the backend selection via the `--quantization-backend` flag, which maps directly to the `QuantizationKind` enum strings:

```bash

# Select FP8-Cast backend

ltx-trainer train \
    --checkpoint /path/to/checkpoint.safetensors \
    --quantization-backend fp8-cast

# Select FP8-Scaled-MM backend

ltx-trainer train \
    --checkpoint /path/to/checkpoint.safetensors \
    --quantization-backend fp8-scaled-mm

```

## Key Implementation Files

The FP8 quantization system spans several packages in the LTX-2 repository:

- **[`packages/ltx-pipelines/src/ltx_pipelines/utils/quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/utils/quantization_factory.py)**: Defines the `QuantizationKind` enum and the `to_policy()` dispatcher that creates `QuantizationPolicy` instances.
- **[`packages/ltx-core/src/ltx_core/quantization/fp8_cast.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_cast.py)**: Implements the FP8-Cast backend, including weight down-casting, scale folding, and optional stochastic rounding.
- **[`packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py)**: Implements the FP8-Scaled-MM backend, providing FP8 linear layers with per-tensor scales and the scaled matrix-multiply kernel.
- **[`packages/ltx-core/src/ltx_core/quantization/policy.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/policy.py)**: Defines the `QuantizationPolicy` dataclass consumed by both backends.
- **[`packages/ltx-trainer/scripts/train.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-trainer/scripts/train.py)**: Entry point that parses `--quantization-backend` arguments and applies the selected policy.

## Summary

- **LTX-2 supports two FP8 quantization backends**: FP8-Cast (up-casting) and FP8-Scaled-MM (scaled matrix-multiply).
- **FP8-Cast** (`fp8-cast`) folds scales into weights during loading and converts FP8 to input dtype at inference, suitable for BF16 checkpoints with `*_scale` tensors.
- **FP8-Scaled-MM** (`fp8-scaled-mm`) maintains separate `weight_scale` tensors and de-quantizes during the matrix operation, requiring native FP8 checkpoints.
- **Configuration** is unified through `QuantizationKind` in [`quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/quantization_factory.py), accessible via both Python API (`QuantizationKind.FP8_CAST.to_policy()`) and CLI (`--quantization-backend`).

## Frequently Asked Questions

### What is the difference between FP8-Cast and FP8-Scaled-MM?

**FP8-Cast** stores weights in FP8 but up-casts them to the input dtype (e.g., BF16) during inference, having folded any scale tensors into the weights at load time. **FP8-Scaled-MM** keeps weights in FP8 and performs a scaled matrix multiplication using separate `weight_scale` tensors, de-quantizing on-the-fly within the kernel. The cast backend requires less specialized kernel support, while the scaled-mm backend offers potentially faster inference on hardware with native FP8 support.

### Which checkpoint format is required for each FP8 backend?

The **FP8-Cast** backend works with BF16 checkpoints that contain pre-quantized `*_scale` tensors, which it folds into the weights during initialization. The **FP8-Scaled-MM** backend requires checkpoints where weights are already stored as FP8 and include corresponding `.weight_scale` tensors for de-quantization. Attempting to use FP8-Scaled-MM with non-FP8 checkpoints will result in runtime errors.

### How do I select a backend using the LTX-2 CLI?

Pass the `--quantization-backend` flag to `ltx-trainer` with the string value `fp8-cast` or `fp8-scaled-mm`. The CLI forwards this string to the `QuantizationKind` dispatcher in [`quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/quantization_factory.py), which instantiates the appropriate `QuantizationPolicy` for the training or inference pipeline.

### Where is the FP8 quantization logic implemented in the source code?

The backend implementations reside in [`packages/ltx-core/src/ltx_core/quantization/fp8_cast.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_cast.py) and [`packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py). The factory pattern and enum definitions are located in [`packages/ltx-pipelines/src/ltx_pipelines/utils/quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/utils/quantization_factory.py), while the policy interface is defined in [`packages/ltx-core/src/ltx_core/quantization/policy.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/policy.py).