How to Apply LoRA Adapters (Distilled, IC-LoRA, Detailing) to LTX-2 Pipelines

You can apply LoRA adapters to LTX-2 pipelines by training an adapter with a task-specific YAML config, saving the lightweight checkpoint, then fusing it into the base model using the fuse_loras.py utility before inference.

LTX-2 supports LoRA (Low-Rank Adaptation) adapters to specialize its core diffusion model for specific tasks without fine-tuning the entire network. The IC-LoRA (In-Context LoRA) variant extends this by conditioning generation on a reference video, learning a lightweight adapter that distills the in-context transformation. This guide walks through the complete workflow based on the Lightricks/LTX-2 source code.

Understanding LTX-2 LoRA Adapter Types

LTX-2 implements several adapter variants, each optimized for different use cases:

• Distilled LoRA — Captures the essence of full-model adaptation in a compact checkpoint
• IC-LoRA — Treats reference videos as conditioning signals for in-context video-to-video transformations
• Detailing LoRA — Enhances specific visual attributes (textures, styles, temporal consistency)

All variants share the same underlying mechanism: trainable low-rank matrices (lora_A, lora_B) that are injected into transformer attention and feed-forward layers.

Step 1: Configure and Train Your LoRA Adapter

LoRA adapter specifications live in YAML configuration files. For video-to-video IC-LoRA, start with the provided template.

Key Configuration File

The packages/ltx-trainer/configs/v2v_ic_lora.yaml file defines:

• Target modules — Which transformer layers receive LoRA (attention, cross-attention, FFN)
• Rank and dropout — Controls adapter capacity and regularization
• Reference conditioning — Enables the in-context learning behavior

Training freezes the base LTX-2 model and optimizes only the LoRA parameters on reference-conditioned pairs (source video + target video).

# Train an IC-LoRA adapter from repository root

!python -m ltx_trainer.train \
    -c packages/ltx-trainer/configs/v2v_ic_lora.yaml \
    --output_dir outputs/ic_lora

The trainer outputs a lightweight checkpoint (.safetensors or .pt) containing only the LoRA matrices—typically megabytes versus gigabytes for full model weights.

Step 2: Fuse LoRA Adapters into the Base Model

Before inference, you must merge the LoRA weights into the base model. The fuse_loras.py utility handles this injection.

The Fusion Utility

Located at packages/ltx-core/src/ltx_core/loader/fuse_loras.py, this script:

• Loads the base LTX-2 model
• Inserts lora_A and lora_B matrices into configured transformer modules
• Computes the low-rank update and adds it to base weights
• Outputs a fused model ready for pipeline execution

# Fuse a single LoRA adapter

python -m ltx_core.loader.fuse_loras \
    --model_path models/ltx2_base \
    --lora_path outputs/ic_lora/checkpoint.safetensors \
    --output_path models/ltx2_ic_fused

Combining Multiple LoRA Adapters

LTX-2 supports stacking multiple adapters for complex generation recipes. The fusion utility merges them sequentially in user-defined order.

# Stack video-to-video IC-LoRA with a style LoRA

python -m ltx_core.loader.fuse_loras \
    --model_path models/ltx2_base \
    --lora_path outputs/ic_lora/checkpoint.safetensors \
    --lora_path outputs/style_lora/checkpoint.safetensors \
    --output_path models/ltx2_stacked_fused

The registry at packages/ltx-core/src/ltx_core/loader/registry.py maps each adapter to its target model components, ensuring proper module alignment during fusion.

Step 3: Run Inference with the Fused Model

The fused model is passed to your chosen pipeline. For IC-LoRA, this requires reference conditioning.

Reference Conditioning in IC-LoRA

Per packages/ltx-pipelines/docs/multimodal-guidance.md, the pipeline:

1. Encodes the reference video through the same encoder used for text prompts
2. Concatenates the reference tensor with diffusion latents
3. Feeds the combined conditioning to the LoRA-augmented model

# Video-to-video generation with fused IC-LoRA

python -m ltx_pipelines.video_to_video \
    --model_path models/ltx2_ic_fused \
    --reference path/to/reference.mp4 \
    --prompt "Turn the daytime scene into a night scene"

The base model remains hardware-agnostic—pipelines run on any profile supported by standard LTX-2 inference.

Complete Workflow Example

# 1️⃣ Train IC-LoRA adapter

!python -m ltx_trainer.train \
    -c packages/ltx-trainer/configs/v2v_ic_lora.yaml \
    --output_dir outputs/ic_lora

# 2️⃣ Fuse into base model

!python -m ltx_core.loader.fuse_loras \
    --model_path models/ltx2_base \
    --lora_path outputs/ic_lora/checkpoint.safetensors \
    --output_path models/ltx2_ic_fused

# 3️⃣ Run video-to-video pipeline with reference conditioning

!python -m ltx_pipelines.video_to_video \
    --model_path models/ltx2_ic_fused \
    --reference path/to/reference.mp4 \
    --prompt "A sunset over the mountains"

Key Implementation Files

File	Purpose
packages/ltx-trainer/configs/v2v_ic_lora.yaml	Training configuration for IC-LoRA adapters
packages/ltx-core/src/ltx_core/loader/fuse_loras.py	LoRA fusion utility for inference preparation
packages/ltx-core/src/ltx_core/loader/registry.py	Adapter-to-module mapping registry
packages/ltx-pipelines/docs/multimodal-guidance.md	Reference conditioning implementation details
packages/ltx-pipelines/docs/pipelines.md	Pipeline selection and execution guide

Summary

• LoRA adapters in LTX-2 are defined via YAML configs and trained with frozen base models
• IC-LoRA enables reference-conditioned generation through in-context learning
• The fuse_loras.py utility injects adapter weights before inference, supporting single or stacked adapters
• Reference conditioning tensors are built from source videos and concatenated with latents
• Fused models execute in standard pipelines without hardware constraints

Frequently Asked Questions

What file format does LTX-2 use for LoRA checkpoints?

LTX-2 saves LoRA checkpoints as .safetensors or .pt files containing only the trainable lora_A and lora_B matrices. These are typically 10-100× smaller than full model weights, making distribution and version control practical.

Can I apply multiple LoRA adapters simultaneously?

Yes. The fuse_loras.py utility accepts multiple --lora_path arguments and merges them sequentially. The order matters—later adapters can override earlier ones. This modularity lets you combine, for example, a video-to-video IC-LoRA with a style-specific detailing LoRA.

Does fusing LoRA adapters modify the original base model?

No. The fusion process creates a new output directory with merged weights; your original base model remains unchanged. This preserves the ability to test different adapter combinations without re-downloading or corrupting core weights.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}