# How to Configure Model Offloading (CPU vs Disk) for Memory-Constrained Systems in LTX-2

> Configure LTX-2 model offloading to CPU or disk for memory-constrained systems. Run massive diffusion models on limited VRAM by streaming weights from system RAM or NVMe storage.

- Repository: [Lightricks/LTX-2](https://github.com/Lightricks/LTX-2)
- Tags: how-to-guide
- Published: 2026-08-20

---

**LTX-2 provides three offloading strategies—`none`, `cpu`, and `disk`—controlled via the `OffloadMode` enum and `--offload` CLI flag, allowing massive diffusion models to run on GPUs with limited VRAM by streaming weights from system RAM or NVMe storage.**

LTX-2 is Lightricks' open-source video diffusion framework, and **model offloading (CPU vs disk) for memory-constrained systems** is a core capability that makes large transformer-based models accessible on consumer hardware. The offloading system is built around a clean abstraction that lets you trade latency for memory capacity without modifying model code.

## Understanding the OffloadMode Enum

The foundation of LTX-2's offloading system is the `OffloadMode` enumeration defined in [`packages/ltx-pipelines/src/ltx_pipelines/utils/types.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/utils/types.py) at line 129. This enum provides three distinct memory tiers:

```python
class OffloadMode(Enum):
    NONE = auto()   # All weights remain on GPU (default)

    CPU = auto()    # Weights stored in system RAM, streamed to GPU per layer

    DISK = auto()   # Weights memory-mapped from NVMe/SSD, minimal RAM footprint

```

Each mode represents a different point on the memory-performance spectrum, and the consistent naming across the codebase makes switching strategies trivial.

## Command-Line Configuration

All LTX-2 pipeline entry points expose the `--offload` flag through the argument parser in [`packages/ltx-pipelines/src/ltx_pipelines/utils/args.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/utils/args.py) at line 643. This unified interface ensures consistent behavior whether you're running text-to-image-to-video, image-to-video, or video-to-video pipelines.

### CPU Offloading

Use **CPU offloading** when GPU VRAM is insufficient but your host machine has ample system RAM (typically 2× the model size):

```bash
python -m ltx_pipelines.ti2vid_one_stage \
    --model_path /path/to/large_model \
    --offload cpu \
    --output_dir ./outputs

```

Under this mode, weight tensors are copied once to CPU memory. During inference, each transformer layer's weights are lazily moved to GPU, executed, and immediately released. This pattern eliminates VRAM pressure at the cost of PCIe bandwidth and higher per-step latency.

### Disk Offloading

Use **disk offloading** when both GPU VRAM and system RAM are constrained, but you have fast NVMe storage available:

```bash
python -m ltx_pipelines.ti2vid_two_stages \
    --model_path /path/to/huge_model \
    --offload disk \
    --output_dir ./outputs

```

Disk offloading leverages memory-mapped weight files. The `StreamingModelBuilder` (detailed below) creates lazy weight objects that read only the necessary tensor slices from disk on demand. Because inactive layers never reside in RAM, you can run models that would otherwise exceed even host memory capacity—ideal for 24GB+ parameter models on laptops or edge devices.

## Python API Configuration

For programmatic use, pass `OffloadMode` directly to pipeline constructors:

```python
from ltx_pipelines.utils.types import OffloadMode
from ltx_pipelines.ti2vid_one_stage import TI2VIDOneStage

# CPU offloading for moderate memory constraints

pipeline_cpu = TI2VIDOneStage(
    model_path="models/large",
    offload_mode=OffloadMode.CPU,
)

# Disk offloading for severe memory constraints

pipeline_disk = TI2VIDOneStage(
    model_path="models/huge",
    offload_mode=OffloadMode.DISK,
)

pipeline_cpu.run()

```

This programmatic approach integrates cleanly with configuration management systems and hyperparameter sweeps.

## How Streaming Blocks Implement Offloading

The actual weight streaming logic lives in [`packages/ltx-pipelines/src/ltx_pipelines/utils/blocks.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/utils/blocks.py). At line 308, the pipeline builder inspects `offload_mode` and conditionally swaps standard transformer blocks for streaming variants:

- When `offload_mode == OffloadMode.NONE`: Standard in-memory blocks are used
- When `offload_mode != OffloadMode.NONE`: `StreamingModelBuilder` constructs blocks with `_cpu` or `_disk` weight backends

Lines 391–394 define the slot bookkeeping system that controls how many weight chunks remain resident simultaneously. For disk offloading specifically, the constant `DISK_CPU_SLOTS` governs this prefetch window, balancing I/O parallelism against memory consumption.

This architecture means offloading is transparent to the model forward pass—the same diffusion code executes regardless of where weights reside.

## Training: Optimizer State Offloading

LTX-2 extends the offloading concept to training workflows through **optimizer state offloading**. Unlike inference weight offloading, this feature moves the optimizer's moment buffers (momentum, variance estimates in Adam/AdamW) off GPU specifically during validation phases.

Control this behavior in [`ltx_trainer/config.py`](https://github.com/Lightricks/LTX-2/blob/main/ltx_trainer/config.py) at line 400:

```python
from ltx_trainer.config import TrainerConfig

cfg = TrainerConfig(
    acceleration={
        "offload_optimizer_during_validation": True,
    },
    # ... other training parameters

)

```

The implementation in [`packages/ltx-trainer/src/ltx_trainer/trainer.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-trainer/src/ltx_trainer/trainer.py) (lines 825–839) provides a context manager that:

1. Detects when entering validation
2. Moves optimizer state tensors to CPU via `optimizer.state_dict()` manipulation
3. Runs the validation loop with reduced GPU memory pressure
4. Restores optimizer state to GPU before resuming training

This is particularly valuable for training large models with AdamW, where optimizer states can consume 2× the model's parameter memory.

## Choosing the Right Offloading Strategy

| Scenario | Recommended Mode | Expected Trade-off |
|----------|---------------|------------------|
| GPU VRAM sufficient for full model | `none` | Maximum speed, no overhead |
| GPU insufficient, RAM ≥ 2× model size | `cpu` | Moderate latency increase, full throughput |
| Both GPU and RAM insufficient, fast NVMe | `disk` | Higher latency, I/O-bound, minimal RAM |
| Training with large optimizer states | `offload_optimizer_during_validation=True` | Slower validation, stable training batch sizes |

## Performance Considerations

- **CPU offloading**: Bandwidth-bound by PCIe speed. Modern PCIe 4.0 x16 links can sustain ~32 GB/s, making this practical for 7B–13B parameter models at modest frame rates.
- **Disk offloading**: Latency-bound by random read IOPS. NVMe SSDs with 500K+ IOPS and high queue depths are essential; SATA SSDs or HDDs will create severe bottlenecks.
- **Slot tuning**: Advanced users can modify `cpu_slots_count` or `DISK_CPU_SLOTS` in [`blocks.py`](https://github.com/Lightricks/LTX-2/blob/main/blocks.py) to increase parallelism at the cost of memory (default configurations target 16GB–24GB consumer GPUs).

## Summary

- **LTX-2 model offloading** is controlled by the `OffloadMode` enum with values `NONE`, `CPU`, and `DISK`
- Use the `--offload` CLI flag or `offload_mode` parameter in Python to select strategies
- CPU offloading streams weights from system RAM; disk offloading uses memory-mapped files for minimal RAM footprint
- The `StreamingModelBuilder` in [`blocks.py`](https://github.com/Lightricks/LTX-2/blob/main/blocks.py) transparently handles weight placement without model code changes
- Training workflows support additional **optimizer state offloading** during validation via `offload_optimizer_during_validation`

## Frequently Asked Questions

### What is the minimum RAM requirement for CPU offloading in LTX-2?

You need system RAM approximately equal to the model's checkpoint size. For a 24GB checkpoint, plan for 24GB RAM plus overhead for activations and operating system. Disk offloading relaxes this to roughly 2–4GB regardless of model size, bounded only by the `DISK_CPU_SLOTS` prefetch window.

### Does disk offloading work with network-attached storage?

Technically yes, but performance will likely be unacceptable. Disk offloading in LTX-2 uses memory-mapped file I/O with random access patterns. Network storage introduces latency that multiplies across thousands of layer accesses per diffusion step. Local NVMe is strongly recommended.

### Can I use optimizer offloading without weight offloading?

Yes. These features operate independently. You can train with full GPU-resident weights (`offload_mode=OffloadMode.NONE`) while still enabling `offload_optimizer_during_validation` to fit larger validation batch sizes. Conversely, you can offload weights for inference without any optimizer involved.

### Where does LTX-2 store temporary files for disk offloading?

The framework memory-maps directly from the model checkpoint path specified via `--model_path`. No additional temporary copies are created, so ensure your checkpoint location has both read bandwidth and sufficient capacity.