# How to Use FP8 Quantization for LTX-2 GPU Memory Reduction

> Reduce LTX-2 GPU memory by 50% using FP8 quantization. Explore FP8-CAST and FP8-SCALED-MM to store model weights in 8-bit format and cut memory usage.

- Repository: [Lightricks/LTX-2](https://github.com/Lightricks/LTX-2)
- Tags: performance
- Published: 2026-08-15

---

**LTX-2 supports two FP8 quantization modes—FP8-CAST and FP8-SCALED-MM—that store model weights in 8-bit floating-point format to cut GPU memory usage by approximately 50% compared to FP16.**

FP8 quantization in Lightricks/LTX-2 reduces the memory footprint of large video generation models by compressing weights to `float8_e4m3fn` format while maintaining inference quality. According to the LTX-2 source code, FP8-CAST performs fused up-casting to BF16 during execution, while FP8-SCALED-MM applies per-tensor scaling factors for more stable numeric behavior. Both modes require CUDA compute capability SM 89 or newer and are explicitly disabled on Apple MPS devices.

## FP8 Quantization Modes Explained

LTX-2 implements two distinct approaches to FP8 quantization, selectable via configuration or command-line arguments.

### FP8-CAST: Fused Up-Casting for Inference

In [`ltx_core/quantization/fp8_cast.py`](https://github.com/Lightricks/LTX-2/blob/main/ltx_core/quantization/fp8_cast.py), the `FP8_CAST` logic (around line 327) stores weights natively in FP8 and up-casts them to BF16 on-the-fly during inference. This casting uses a fused add-round kernel that minimizes memory movement.

Typical characteristics:
- **Memory savings**: ~31 GiB for full-size models (~50% reduction vs. FP16)
- **Compute pattern**: FP8 storage → BF16 compute → FP8 gradients (if training)
- **Best for**: Pre-quantized checkpoints optimized for cast-based loading

### FP8-SCALED-MM: Scaled Matrix Multiplication

The `FP8Linear` class in [`ltx_core/quantization/fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/ltx_core/quantization/fp8_scaled_mm.py) (lines 23-52) stores both FP8 weights and per-tensor or per-block scaling factors. During GEMM operations, weights are de-quantized using these scales before multiplication.

Typical characteristics:
- **Memory savings**: Similar to FP8-CAST (~50%)
- **Numeric stability**: More stable for some model architectures due to explicit scaling
- **Best for**: Checkpoints requiring fine-grained scale control

In [`ltx_pipelines/utils/quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/ltx_pipelines/utils/quantization_factory.py) (lines 23-39), the string identifiers `"fp8-cast"` and `"fp8-scaled-mm"` map to the internal `QuantizationKind` enum.

## Enabling FP8 Quantization

### Configuration via YAML

Set `weight_precision` in your inference configuration:

```yaml

# config_fp8_cast.yaml

model:
  name: ltx-2-base
  checkpoint: /path/to/checkpoint.pt
inference:
  weight_precision: fp8-cast
  device: cuda

```

```yaml

# config_fp8_scaled_mm.yaml

inference:
  weight_precision: fp8-scaled-mm

```

### Command-Line Usage

The [`serve_captioner.py`](https://github.com/Lightricks/LTX-2/blob/main/serve_captioner.py) script uses FP8 by default (line 11). Override explicitly:

```bash
python -m ltx_trainer.scripts.serve_captioner \
  --checkpoint /models/ltx-2.pt \
  --weight-precision fp8-cast

```

CLI help text in [`ltx_pipelines/utils/args.py`](https://github.com/Lightricks/LTX-2/blob/main/ltx_pipelines/utils/args.py) (lines 679-680) documents these options:

```bash
python -m ltx_trainer.scripts.serve_captioner --help

# Output includes: weight_precision choices: fp8, fp8-cast, fp8-scaled-mm, ...

```

### Programmatic API

```python
from ltx_core.quantization import QuantizationKind
from ltx_pipelines.utils.quantization_factory import build_quantizer

# Instantiate FP8-CAST quantizer

quantizer = build_quantizer(
    kind=QuantizationKind.FP8_CAST,
    checkpoint_path="models/ltx-2.pt"
)

# Apply to loaded model

model = load_my_ltx2_model()
quantizer.apply(model)

```

## Verifying GPU Memory Reduction

Measure weight memory before and after quantization:

```python
import torch

def measure_weight_memory(model):
    return sum(
        p.numel() * p.element_size() 
        for p in model.parameters()
    ) / 1e9

model = load_my_ltx2_model()
print(f"FP16 weight memory: {measure_weight_memory(model):.2f} GB")

# Apply FP8 quantization

from ltx_pipelines.utils.quantization_factory import build_quantizer
quantizer = build_quantizer(kind="fp8-cast")
quantizer.apply(model)

print(f"FP8 weight memory: {measure_weight_memory(model):.2f} GB")

# Expected: ~50% reduction

```

## Hardware and Compatibility Requirements

### CUDA Architecture Requirements

The [`ltx_kernels/blockwise/linear.py`](https://github.com/Lightricks/LTX-2/blob/main/ltx_kernels/blockwise/linear.py) module dispatches to architecture-specific kernels:

| Architecture | Compute Capability | FP8 Support |
|-------------|-------------------|-------------|
| NVIDIA Ada Lovelace | SM 89 | Full |
| NVIDIA Hopper | SM 90 | Full |
| Pre-Ada GPUs | SM 80 and below | Not supported |

### Device Guard in quantization.py

The validation logic in [`packages/ltx-trainer/src/ltx_trainer/quantization.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-trainer/src/ltx_trainer/quantization.py) (line 189) explicitly raises an error for non-CUDA devices:

```python

# From quantization.py (line 189)

if device.type == "mps":
    raise RuntimeError("FP8 quantization is not supported on MPS devices")

```

When FP8 is unavailable, the system falls back to INT8 or INT4 quantization alternatives.

### Checkpoint Compatibility

Some LTX-2 checkpoints are pre-quantized for specific modes. Check your checkpoint documentation:

- Checkpoints with `.fp8-cast.pt` suffix → Use `fp8-cast`
- Checkpoints with `.fp8-scaled.pt` suffix → Use `fp8-scaled-mm`
- Generic `.pt` checkpoints → Either mode may work; test for numeric stability

## Key Source Files for FP8 Quantization

| Path | Purpose |
|------|---------|
| [`packages/ltx-trainer/src/ltx_trainer/quantization.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-trainer/src/ltx_trainer/quantization.py) | Device validation and fallback logic |
| [`packages/ltx-pipelines/src/ltx_pipelines/utils/quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/utils/quantization_factory.py) | String-to-enum mapping for FP8 modes |
| [`packages/ltx-pipelines/src/ltx_pipelines/utils/args.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/utils/args.py) | CLI argument definitions |
| [`packages/ltx-core/src/ltx_core/quantization/fp8_cast.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_cast.py) | FP8-CAST implementation (~line 327) |
| [`packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-core/src/ltx_core/quantization/fp8_scaled_mm.py) | FP8Linear and scaled MM (lines 23-52) |
| [`packages/ltx-kernels/src/ltx_kernels/blockwise/linear.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-kernels/src/ltx_kernels/blockwise/linear.py) | CUDA kernel dispatch for SM 89/90 |

## Summary

- **FP8-CAST** stores weights in `float8_e4m3fn` and up-casts to BF16 during inference via fused kernels in [`fp8_cast.py`](https://github.com/Lightricks/LTX-2/blob/main/fp8_cast.py)
- **FP8-SCALED-MM** adds per-tensor scaling factors for de-quantization during GEMM, implemented in [`fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/fp8_scaled_mm.py)
- Both modes reduce GPU memory by ~50% compared to FP16, from ~62 GiB to ~31 GiB for full LTX-2 models
- Activation via `weight_precision: fp8-cast` or `fp8-scaled-mm` in YAML configs, or `--weight-precision` CLI flag
- Requires CUDA SM 89+ (Ada Lovelace/Hopper); MPS devices are blocked by explicit guard in [`quantization.py`](https://github.com/Lightricks/LTX-2/blob/main/quantization.py)

## Frequently Asked Questions

### What GPUs support FP8 quantization in LTX-2?

NVIDIA Ada Lovelace (RTX 40-series, SM 89) and Hopper (H100, SM 90) GPUs support FP8 natively. The [`ltx_kernels/blockwise/linear.py`](https://github.com/Lightricks/LTX-2/blob/main/ltx_kernels/blockwise/linear.py) module dispatches architecture-specific kernels based on detected compute capability. Older GPUs (Ampere and earlier) lack native FP8 hardware support and will raise errors or fall back to INT quantization.

### How do I choose between FP8-CAST and FP8-SCALED-MM?

Select **FP8-CAST** for checkpoints pre-optimized for cast-based loading and when maximum inference speed is prioritized. Choose **FP8-SCALED-MM** when your checkpoint includes scale factors or when you observe numeric instability with pure casting. The [`quantization_factory.py`](https://github.com/Lightricks/LTX-2/blob/main/quantization_factory.py) module validates that your selected mode matches checkpoint requirements.

### Can I train with FP8 quantization enabled?

The source code shows FP8 is primarily implemented for **inference weight storage**, with `training.precision` typically remaining at FP16 or BF16 for gradient computation. The `FP8Linear` modules handle weight gradients carefully—check [`fp8_scaled_mm.py`](https://github.com/Lightricks/LTX-2/blob/main/fp8_scaled_mm.py) for training-specific behavior if fine-tuning quantized models.

### Why does FP8 quantization fail on my Mac?

The guard clause in [`quantization.py`](https://github.com/Lightricks/LTX-2/blob/main/quantization.py) (line 189) explicitly raises `RuntimeError` for MPS devices because Apple's Metal Performance Shaders lack native FP8 support. Use INT8 or INT4 quantization alternatives, or run on CUDA hardware.