How to Debug Attention Patterns and Guidance Issues in LTX-2: A Complete Technical Guide

To debug attention patterns and guidance issues in LTX-2, extract and visualize the per-block keep-mask from BatchedPerturbationConfig, isolate specific attention types using PerturbationConfig, and force specific attention backends when the automatic selector misbehaves.

LTX-2 from Lightricks is a multi-modal diffusion model built on a Transformer-based backbone with fine-grained control over attention patterns. Understanding how to inspect and manipulate these patterns is essential when diagnosing why Spatio-Temporal Guidance (STG) or cross-modal guidance isn't producing expected results. This article walks through the precise mechanisms and code paths you need.

---

Understanding the Perturbation System Architecture

LTX-2's debugging capabilities center on a perturbation primitive system that allows selective disabling of attention layers per-block and per-modality. This system lives in packages/ltx-core/src/ltx_core/guidance/perturbations.py.

Core Data Structures

The four fundamental perturbation types are defined in PerturbationType (lines 13-16 of perturbations.py):

Enum Value	Purpose
SKIP_VIDEO_SELF_ATTN	Disable video self-attention
SKIP_AUDIO_SELF_ATTN	Disable audio self-attention
SKIP_A2V_CROSS_ATTN	Disable audio→video cross-attention
SKIP_V2A_CROSS_ATTN	Disable video→audio cross-attention

Each Perturbation (lines 19-25) pairs a type with an optional list of blocks—None means apply to all blocks. PerturbationConfig (lines 36-43) aggregates these for a single sample, while BatchedPerturbationConfig (lines 70-86) converts per-sample configs into a dense tensor _block_masks with shape [type, block, sample].

The mask values are binary: 1 = keep attention, 0 = perturb (skip). This tensor is built once outside compiled regions and read as a regular tensor during every forward pass.

---

How the Transformer Applies Perturbation Masks

The actual mask application happens in packages/ltx-core/src/ltx_core/model/transformer/attention.py.

In Attention.forward (lines 30-55), the module receives and forwards the mask to either self.attention_function or self.masked_attention_function. Critically, lines 72-74 apply the mask after attention computation:

out = out * perturbation_mask + v * (1 - perturbation_mask)

If the mask is all-zeros, the query/key path is skipped entirely (lines 61-63), providing a clean diagnostic signal.

---

Extracting and Inspecting Attention Masks

The first step in debugging is retrieving the actual mask values the model uses. Here's how to extract the keep-mask for a specific block and attention type:

import torch
from ltx_core.model.transformer.attention import Attention, AttentionOps
from ltx_core.guidance.perturbations import (
    BatchedPerturbationConfig, PerturbationConfig, Perturbation, PerturbationType
)

# Build config that skips block 5 video self-attention only

pert = Perturbation(type=PerturbationType.SKIP_VIDEO_SELF_ATTN, blocks=[5])
cfg = PerturbationConfig(perturbations=[pert])
batched_cfg = BatchedPerturbationConfig(
    [cfg], num_blocks=12, device="cpu"  # 12 = total transformer blocks

)

# Create dummy attention layer

attn = Attention(query_dim=64, heads=4, dim_head=32, ops=AttentionOps())
x = torch.randn(1, 10, 64)  # (B, T, C)

# Extract mask for block 5

mask = batched_cfg.mask(PerturbationType.SKIP_VIDEO_SELF_ATTN, block=5)

# Run forward and inspect

out = attn(x, mask=mask)
print("Mask for block 5 (video self-attn):", mask.squeeze().numpy())

Key diagnostic signal: The printed mask shows 0 for skipped blocks, 1 elsewhere. If you see unexpected values, the BatchedPerturbationConfig construction is the source of your bug.

---

Isolating Cross-Modal Guidance Issues

Cross-modal attention problems require targeted perturbation of audio-video interactions. The validation_runner.py shows how _build_modality_perturbation_config (lines 1301-1306) creates these perturbations programmatically.

Manual isolation follows the same pattern:

# Disable video→audio cross-attention across all blocks

cross_pert = Perturbation(
    type=PerturbationType.SKIP_V2A_CROSS_ATTN, 
    blocks=None  # None = all blocks

)
cross_cfg = PerturbationConfig(perturbations=[cross_pert])
batched_cross = BatchedPerturbationConfig(
    [cross_cfg], num_blocks=12, device="cpu"
)

# Apply to any block (here block 7)

cross_mask = batched_cross.mask(
    PerturbationType.SKIP_V2A_CROSS_ATTN, 
    block=7
)
out = attn(x, context=x, mask=cross_mask)

Compare outputs with and without this mask to determine if video→audio attention is driving your guidance issue.

---

Visualizing Spatial Conditioning Masks

When using --conditioning-attention-mask from the CLI (defined in packages/ltx-pipelines/src/ltx_pipelines/utils/args.py, lines 427-435), you can extract and visualize the spatial mask:

import numpy as np
import matplotlib.pyplot as plt
from ltx_pipelines.utils.args import _load_mask_video

# Load mask with strength scaling

mask_path = "mask.mp4"
strength = 0.6
mask_tensor = _load_mask_video(mask_path) * strength  # (T, H, W)

# Visualize first frame

frame = mask_tensor[0].cpu().numpy()
plt.imshow(frame, cmap="viridis")
plt.title("Conditioning attention mask (frame 0)")
plt.colorbar(label="attention strength")
plt.axis("off")
plt.show()

Higher values in brighter regions indicate increased sensitivity to conditioning prompts. Misaligned masks here directly explain spatial guidance failures.

---

Forcing Specific Attention Backends

LTX-2 automatically selects between Flash Attention 3/4, cuDNN SDPA, and Apple MPS kernels (lines 9-16 and 98-108 of attention.py). When debugging, this automatic selection can mask kernel-specific bugs.

Force a deterministic backend like this:

from ltx_core.model.transformer.attention import (
    AttentionOps, AttentionFunction, MaskedAttentionFunction
)

ops = AttentionOps(
    attention_function=AttentionFunction.PYTORCH.to_callable(),
    masked_attention_function=MaskedAttentionFunction.PYTORCH.to_callable(),
)
attn = Attention(query_dim=64, heads=4, dim_head=32, ops=ops)

Use case: If masking works with PyTorch SDPA but fails with Flash Attention, you've isolated a kernel-level mask handling bug.

---

Key Source Files Reference

File	Purpose	Critical Lines
perturbations.py	Mask tensor construction	13-16 (types), 70-86 (batching)
attention.py	Mask application, backend selection	30-55 (forward), 61-63 (skip logic), 72-74 (mask apply)
validation_runner.py	STG and cross-modal config builders	1285-1293 (STG), 1301-1306 (cross-modal)
args.py	CLI mask loading utilities	427-435
trainer.py	Model invocation with perturbations	379-387
model.py	Empty config injection	Transformer forward wrapper

---

Summary

Debugging attention patterns in LTX-2 follows a systematic workflow:

• Inspect masks via BatchedPerturbationConfig.mask() to verify binary keep/skip values
• Isolate specific attention types using targeted Perturbation configs with precise block lists
• Visualize spatial conditioning masks to confirm correct loading and strength scaling
• Pin backends to PyTorch SDPA when automatic selection obscures kernel-level bugs
• Compare outputs with and without perturbations to establish causality

The perturbation system's clean separation between configuration (perturbations.py), application (attention.py), and pipeline integration (validation_runner.py) enables precise, block-level diagnosis of guidance issues.

---

Frequently Asked Questions

How do I verify that a specific attention block is actually being skipped?

Create a Perturbation with blocks=[target_block], extract the mask via BatchedPerturbationConfig.mask(), and confirm it contains zeros. Then run identical forward passes with and without the perturbation—the outputs should differ measurably. If they don't, verify the mask is being passed correctly through the Attention.forward signature in attention.py.

Can I skip multiple attention types simultaneously in the same block?

Yes. A single PerturbationConfig can contain multiple Perturbation objects with different PerturbationType values targeting the same or different blocks. The BatchedPerturbationConfig will merge these into separate channels of the _block_masks tensor.

Why would I disable the automatic attention backend selector?

The automatic selector in attention.py (lines 98-108) prioritizes speed and may fall back silently when mask shapes are unsupported. This fallback can change timing and occasionally bypass mask handling. Forcing AttentionFunction.PYTORTO makes behavior deterministic and easier to step through with debuggers like pdb or PyTorch's anomaly detection.

How do spatial conditioning masks interact with STG perturbations?

They compose multiplicatively. The spatial mask from --conditioning-attention-mask is downsampled and applied as an additional modulation factor to the block-level perturbation mask. Debugging both together requires visualizing the spatial mask (as in section 3.3) and verifying the block-level mask separately—issues in either layer can cause unexpected guidance behavior.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}