# Requirements for FP8 Scaled MM Quantization in LTX-2: A Complete Guide

> Learn the requirements for FP8 scaled MM quantization in LTX-2. Use FP8 weights, per-tensor scales, and a PyTorch build with torch.float8_e4m3fn for efficient quantization.

- Repository: [Lightricks/LTX-2](https://github.com/Lightricks/LTX-2)
- Tags: how-to-guide
- Published: 2026-06-21

---

**LTX-2 requires pre-quantized checkpoints containing FP8 weights (`F8_E4M3` dtype) paired with per-tensor scale tensors, plus a PyTorch build supporting `torch.float8_e4m3fn`, to utilize FP8 scaled-matrix-multiply (MM) quantization.**

The LTX-2 video generation framework from Lightricks provides an optimized inference path using FP8 scaled MM quantization, but this capability depends on specific checkpoint characteristics and environment requirements. As implemented in the `ltx_core.quantization` module, this quantization method differs from runtime casting approaches by requiring weights that are already quantized to the `F8_E4M3` format with pre-calculated scaling factors.

## Prerequisites for FP8 Scaled MM Quantization

### Pre-quantized Checkpoint Structure

The FP8 scaled MM path in LTX-2 strictly requires checkpoints that have been quantized ahead of time. According to the source code in [`packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py), your checkpoint must satisfy two structural conditions:

1. **FP8 weight tensors** – The safetensors file must contain weights with dtype `F8_E4M3` (mapped to `torch.float8_e4m3fn` in PyTorch).
2. **Per-tensor scale tensors** – For every `*.weight` tensor, there must be a corresponding `*.weight_scale` tensor in the same checkpoint file.

The loader discovers these pairs through the `get_fp8_swap_module_ops` function (lines 50-61). If no valid weight/scale pairs are detected, the function raises a `ValueError` (lines 62-66), preventing ambiguous interpretation with BF16 checkpoints.

### PyTorch Version Compatibility

Your environment must run a recent PyTorch build that supports the `torch.float8_e4m3fn` dtype. The `FP8Linear` layer constructor (lines 30-44 in [`fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/fp8_scaled_mm.py)) stores weights directly in this format, requiring native FP8 tensor support in your PyTorch installation.

### Optional TensorRT-LLM Acceleration

While not strictly required, the implementation automatically detects and utilizes TensorRT-LLM kernels when available. The `trtllm_scaled_mm_usable()` function (lines 53-64) checks for this optimization. If TensorRT-LLM is unavailable, the code falls back to PyTorch's native `_scaled_mm` implementation (lines 66-80).

## Verification and Implementation

### Checking Checkpoint Compatibility

Before loading a model, verify that your checkpoint supports FP8 scaled MM quantization by attempting to discover the swap operations:

```python
from ltx_core.quantization.fp8_scaled_mm import get_fp8_swap_module_ops

try:
    ops = get_fp8_swap_module_ops("/path/to/checkpoint.safetensors")
    print("Checkpoint is FP8-compatible – FP8Linear layers will be swapped.")
except ValueError as e:
    print("Incompatible checkpoint:", e)

```

This validation ensures that the required `*.weight_scale` tensors exist alongside the FP8 weights, preventing runtime errors during model initialization.

### Loading Models with FP8 Quantization Policy

To apply FP8 scaled MM quantization, construct a quantization policy using the builder function and pass it to the model loader:

```python
from ltx_core.quantization.fp8_scaled_mm import build_policy
from ltx_core.loader.sft_loader import load_model

checkpoint = "models/ltx-2-19b-dev-fp8.safetensors"

# Build the quantization policy

policy = build_policy(checkpoint)

# Load model with FP8 scaled MM support

model = load_model(
    checkpoint_path=checkpoint,
    quant_policy=policy,
)

```

The `FP8Linear` layers will automatically handle the scaled matrix multiplication during the forward pass, selecting the optimal kernel based on your hardware capabilities.

### Alternative Path for BF16 Checkpoints

If your checkpoint contains BF16 weights without pre-calculated `*.weight_scale` tensors, you cannot use the scaled MM path. Instead, utilize the FP8 cast implementation available in [`packages/ltx-core/src/ltx_core/quantization/fp8_cast.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_cast.py), which performs quantization at runtime rather than relying on pre-quantized weights.

## Summary

- **Pre-quantized weights required**: Checkpoints must contain `F8_E4M3` dtype tensors with matching `*.weight_scale` tensors.
- **Discovery mechanism**: The `get_fp8_swap_module_ops` function validates checkpoint compatibility by scanning for weight/scale pairs.
- **PyTorch dependency**: Requires a PyTorch version supporting `torch.float8_e4m3fn` for the `FP8Linear` implementation.
- **Performance optimization**: Automatically uses TensorRT-LLM kernels when available, falling back to PyTorch `_scaled_mm` otherwise.
- **BF16 alternative**: Use the [`fp8_cast.py`](https://github.com/Lightricks/LTX-2/blob/main/fp8_cast.py) module for checkpoints that lack pre-quantized FP8 weights.

## Frequently Asked Questions

### What happens if I try to use FP8 scaled MM with a BF16 checkpoint?

The `get_fp8_swap_module_ops` function will raise a `ValueError` because the absence of `*.weight_scale` tensors makes the quantization configuration ambiguous. For BF16 checkpoints, you must use the FP8 cast path ([`fp8_cast.py`](https://github.com/Lightricks/LTX-2/blob/main/fp8_cast.py)) instead, which handles quantization during inference rather than expecting pre-quantized weights.

### Do I need TensorRT-LLM installed to use FP8 scaled MM quantization?

No, TensorRT-LLM is optional. The [`fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/fp8_scaled_mm.py) implementation checks `trtllm_scaled_mm_usable()` and automatically falls back to PyTorch's native `_scaled_mm` function (lines 66-80) if TensorRT-LLM is unavailable. However, having TensorRT-LLM installed provides faster inference through optimized kernels.

### What is the difference between `build_policy` and `get_fp8_swap_module_ops`?

The `get_fp8_swap_module_ops` function performs the low-level validation, scanning the checkpoint for `F8_E4M3` weights and their corresponding scale tensors (lines 50-61). The `build_policy` function wraps this discovery process to create a `QuantizationPolicy` object that the model loader in [`packages/ltx-core/src/ltx_core/loader/sft_loader.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/loader/sft_loader.py) can consume to initialize `FP8Linear` layers throughout the network.

### Where are the FP8 weights stored in the model architecture?

The `FP8Linear` class (defined in [`fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/fp8_scaled_mm.py), lines 30-44) replaces standard `Linear` layers when the quantization policy is applied. These layers store weights in `torch.float8_e4m3fn` format and apply the per-tensor scales during the forward pass using either TensorRT-LLM or PyTorch scaled matrix multiplication kernels.