# How to Apply IC-LoRA for Video-to-Video Transformations Using LTX-2

> Learn to apply IC-LoRA for video-to-video transformations using LTX-2. Train a LoRA with VideoToVideoStrategy and run inference through ICLoraPipeline for stunning results.

- Repository: [Lightricks/LTX-2](https://github.com/Lightricks/LTX-2)
- Tags: how-to-guide
- Published: 2026-08-14

---

**IC-LoRA for video-to-video transformations in LTX-2 requires training a LoRA with the `VideoToVideoStrategy` and running inference through the `ICLoraPipeline` with reference video conditioning.**

LTX-2 implements **In-Context LoRA (IC-LoRA)** through a specialized two-stage architecture that enables powerful video-to-video (V2V) transformations. This guide walks through the complete workflow—from training configuration to inference—based on the official [Lightricks/LTX-2](https://github.com/Lightricks/LTX-2) source code.

## Architecture Overview

IC-LoRA in LTX-2 splits responsibilities between training and inference components:

| Component | Purpose | Source Location |
|-----------|---------|---------------|
| `VideoToVideoStrategy` | Training-side batch processing, reference/target latent concatenation, and conditioning mask construction | [`packages/ltx-trainer/src/ltx_trainer/training_strategies/video_to_video.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-trainer/src/ltx_trainer/training_strategies/video_to_video.py) |
| `ICLoraPipeline` | Inference-side two-stage generation with distilled transformer | [`packages/ltx-pipelines/src/ltx_pipelines/ic_lora.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/ic_lora.py) |
| `append_ic_lora_reference_video_conditionings()` | Converts video paths into structured conditioning items with automatic down-sampling | [`packages/ltx-pipelines/src/ltx_pipelines/iclora_utils.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/iclora_utils.py) |
| `read_lora_reference_downscale_factor()` / `read_lora_reference_temporal_scale_factor()` | Extract scaling metadata from LoRA safetensors for consistent resizing | [`packages/ltx-pipelines/src/ltx_pipelines/iclora_utils.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/iclora_utils.py) |

## Training IC-LoRA for Video-to-Video

### Dataset Requirements

Each training sample must contain a reference video latent directory (`reference_latents/`). The dataset layout follows the standard LTX-2 structure with this additional subdirectory.

### Training Flow

The `VideoToVideoStrategy.prepare_training_inputs` method handles five critical operations:

1. **Loads reference latents** (`ref_latents`) alongside target latents
2. **Infers down-scale factor**—the ratio of reference resolution to target resolution
3. **Builds concatenated latent tensor** `[ref, noisy_target]`
4. **Constructs conditioning masks**—reference tokens always use timestep 0; target tokens may condition on first frame with probability `first_frame_conditioning_p`
5. **Computes loss only on target portion**—reference latents remain untouched

This design forces the model to learn style/structure transfer rather than reconstruction.

### Configuration

Use the example training config at [`packages/ltx-trainer/configs/v2v_ic_lora.yaml`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-trainer/configs/v2v_ic_lora.yaml). Key parameters include:

- `first_frame_conditioning_p`: Probability of conditioning target on reference first frame
- Reference latent paths resolved relative to each sample

## Inference with IC-LoRA for Video-to-Video

IC-LoRA inference requires **distilled model checkpoints**. Standard full-model pipelines do not support this feature.

### Setup and Initialization

```python
from ltx_pipelines import ICLoraPipeline
from ltx_pipelines.utils.model_paths import ModelPaths

# Initialize with distilled transformer and upsampler

model_paths = ModelPaths(
    transformer="models/ltx-2.5/diffusion_models/ltx-2.5-22b-distilled-transformer-bf16.safetensors",
    video_vae="models/ltx-2.5/vae/video_vae.safetensors",
    audio_vae="models/ltx-2.5/vae/audio_vae.safetensors",
    spatial_upsampler="models/ltx-2.5/latent_upscale_models/ltx-2.5-latent-spatial-upscaler-x2-bf16-1.0.safetensors",
)

# LoRA must contain reference_downscale_factor in metadata

lora_paths = [
    "models/ltx-2.5/loras/ltx-2.5-22b-ic-lora-video.safetensors"
]

pipeline = ICLoraPipeline(
    model_paths=model_paths,
    spatial_upsampler_path=model_paths.spatial_upsampler,
    loras=lora_paths,
)

```

The `ICLoraPipeline.__init__` automatically calls `read_lora_reference_downscale_factor()` and `read_lora_reference_temporal_scale_factor()` to extract scaling metadata. Conflicting factors across multiple LoRAs raise an error.

### Generation with Reference Video

```python

# Reference video provides style/structure to transfer

video_conditioning = [
    ("assets/reference_style.mp4", 1.0)  # (path, strength)

]

frames_iter, audio, tiling_cfg = pipeline(
    prompt="A sunrise over a futuristic city, in the style of the reference video",
    seed=12345,
    height=512,
    width=768,
    num_frames=121,        # Must satisfy 8k+1 convention

    frame_rate=30.0,
    images=[],             # No image conditioning

    video_conditioning=video_conditioning,
    enhance_prompt=False,
)

generated_frames = list(frames_iter)

```

The pipeline executes:

1. **Stage 1**: Distilled transformer denoises concatenated `[ref, noisy_target]` latents using `VideoConditionByReferenceLatent` items
2. **Stage 2**: Spatial upsampler (2×) refines to final resolution

The `append_ic_lora_reference_video_conditionings` helper in [`iclora_utils.py`](https://github.com/Lightricks/LTX-2/blob/main/iclora_utils.py) automatically down-samples the reference video to match target dimensions using the stored `reference_downscale_factor`.

### Spatial Masking for Selective Application

Control where IC-LoRA influence applies using attention masks:

```python
import torch

# Binary mask: True = apply reference conditioning

mask = torch.zeros((height, width), dtype=torch.bool)
mask[100:400, 200:600] = True  # Center region only

frames_iter, audio, tiling_cfg = pipeline(
    prompt="Futuristic city with selective style transfer",
    seed=12345,
    height=512,
    width=768,
    num_frames=121,
    video_conditioning=[("assets/reference_style.mp4", 1.0)],
    conditioning_attention_mask=mask,
    conditioning_attention_strength=0.8,  # Scale mask influence

)

```

The `downsample_mask_video_to_latent` utility (also in [`iclora_utils.py`](https://github.com/Lightricks/LTX-2/blob/main/iclora_utils.py)) handles resolution matching automatically.

## Key Implementation Files

| Path | Description |
|------|-------------|
| [`packages/ltx-trainer/src/ltx_trainer/training_strategies/video_to_video.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-trainer/src/ltx_trainer/training_strategies/video_to_video.py) | `VideoToVideoStrategy` class with `prepare_training_inputs()` method |
| [`packages/ltx-pipelines/src/ltx_pipelines/ic_lora.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/ic_lora.py) | `ICLoraPipeline` two-stage inference implementation |
| [`packages/ltx-pipelines/src/ltx_pipelines/iclora_utils.py`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-pipelines/src/ltx_pipelines/iclora_utils.py) | Metadata readers, mask utilities, conditioning helpers |
| [`packages/ltx-trainer/configs/v2v_ic_lora.yaml`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-trainer/configs/v2v_ic_lora.yaml) | Example training configuration |
| [`packages/ltx-trainer/docs/training-modes.md`](https://github.com/Lightricks/LTX-2/blob/main/packages/ltx-trainer/docs/training-modes.md) | IC-LoRA training mode documentation |

## Summary

- **IC-LoRA enables video-to-video style transfer** by conditioning generation on clean reference latents
- **Training requires** `VideoToVideoStrategy` with reference latent directories and proper metadata storage
- **Inference requires** `ICLoraPipeline` with distilled models and LoRAs containing `reference_downscale_factor`
- **Reference scaling is automatic**—the pipeline reads metadata and applies consistent spatial/temporal subsampling
- **Spatial control** available via `conditioning_attention_mask` for selective region application

## Frequently Asked Questions

### What model type does IC-LoRA require?

IC-LoRA requires **distilled transformer checkpoints** only. The two-stage `ICLoraPipeline` is incompatible with standard full-model pipelines. Use paths like `ltx-2.5-22b-distilled-transformer-bf16.safetensors`.

### How is the reference video resolution matched to the target?

The `read_lora_reference_downscale_factor()` function extracts the ratio from LoRA safetensors metadata. The reference is down-sampled to `height // downscale_factor` by `width // downscale_factor` before latent encoding.

### Can I use multiple reference videos?

Yes. Pass multiple tuples to `video_conditioning`: `[("ref1.mp4", 0.8), ("ref2.mp4", 0.5)]`. Each becomes a separate `VideoConditionByReferenceLatent`. Ensure all referenced LoRAs share identical down-scale and temporal scale factors.

### What frame count should I use for generation?

LTX-2 requires `num_frames = 8k + 1` (e.g., 121, 129, 137). The temporal pipeline architecture enforces this constraint. Frame rates are flexible but 24-30 fps works best with most trained LoRAs.