# How to Integrate TriAttention KV Cache Compression with LongLive: Complete Implementation Guide

> Learn to integrate TriAttention KV cache compression with LongLive. Reduce GPU memory by 50% with this guide and boost your model's efficiency without sacrificing quality.

- Repository: [NVIDIA Research Projects/LongLive](https://github.com/NVlabs/LongLive)
- Tags: how-to-guide
- Published: 2026-05-24

---

**You integrate TriAttention KV cache compression with LongLive by setting `use_tri_attn_kv_compression: true` in your configuration and installing the external TriAttention library, which transparently swaps default KV-cache tensors with compressed representations that reduce GPU memory usage by approximately 50% while preserving generation quality.**

LongLive 2.0 introduces native support for **TriAttention KV cache compression**, enabling significant memory optimization during video generation without altering the inference API. This integration compresses key-value tensors during cache updates and decompresses them on-the-fly during attention calculations. The following guide details the exact steps to activate and implement this optimization using the NVlabs/LongLive codebase.

## How TriAttention KV Cache Compression Works in LongLive

The integration intercepts KV-cache operations within the causal transformer implementation ([`wan_5b/modules/causal_model.py`](https://github.com/NVlabs/LongLive/blob/main/wan_5b/modules/causal_model.py)). When the configuration flag `use_tri_attn_kv_compression` is enabled, the system:

1. **Compresses on storage**: During cache updates, the model invokes `triattention.compress_kv_cache()` to store quantized representations instead of full-precision tensors.
2. **Decompresses on retrieval**: During attention calculations, `triattention.decompress_kv_cache()` restores the full keys and values temporarily for computation.
3. **Maintains API compatibility**: The surrounding pipeline continues to pass a standard `kv_cache` dictionary, ensuring that multi-shot inference and async decoding workflows require no modifications.

This compression is **lossless up to the quantization level**, delivering roughly **50% memory savings** without perceptible quality degradation.

## Prerequisites and Installation

Before enabling compression, install the TriAttention library from the dedicated LongLive branch:

```bash
pip install git+https://github.com/WeianMao/triattention.git@longlive

```

Verify the installation by confirming LongLive can import the compression utilities:

```python
from triattention import compress_kv_cache, decompress_kv_cache

```

## Enabling Compression in Your Configuration

You can activate TriAttention KV cache compression either through YAML configuration or programmatically before pipeline initialization.

### YAML Configuration Method

Add the compression flag to your [`configs/inference.yaml`](https://github.com/NVlabs/LongLive/blob/main/configs/inference.yaml):

```yaml
model:
  use_tri_attn_kv_compression: true

```

### Programmatic Activation Method

Modify the configuration object dynamically before building the pipeline:

```python
from utils.config import normalize_config
from omegaconf import OmegaConf
from pipeline import CausalDiffusionInferencePipeline

cfg = OmegaConf.load("configs/inference.yaml")
cfg.model.use_tri_attn_kv_compression = True
config = normalize_config(cfg)

pipe = CausalDiffusionInferencePipeline(config, device="cuda")

```

## Source Code Implementation Details

The core integration resides in [`wan_5b/modules/causal_model.py`](https://github.com/NVlabs/LongLive/blob/main/wan_5b/modules/causal_model.py), where the model conditionally applies compression based on the configuration flag.

During the KV-cache update step (`_update_cache_and_get_kv`):

```python
if self.cfg.use_tri_attn_kv_compression:
    kv_cache = compress_kv_cache(kv_cache)

```

During attention calculation (`_forward_with_cache`):

```python
if self.cfg.use_tri_attn_kv_compression:
    k, v = decompress_kv_cache(kv_cache)

```

The sequence-parallel variant in [`wan_5b/modules/causal_model_sp_ulysses.py`](https://github.com/NVlabs/LongLive/blob/main/wan_5b/modules/causal_model_sp_ulysses.py) implements identical logic, ensuring consistency across distributed training configurations. Additionally, [`utils/quant.py`](https://github.com/NVlabs/LongLive/blob/main/utils/quant.py) provides de-quantization utilities utilized during the decompression phase to restore tensor precision.

## Complete End-to-End Integration Example

The following example demonstrates how to integrate TriAttention KV cache compression with LongLive for video generation:

```python
import torch
from omegaconf import OmegaConf
from utils.inference_utils import (
    load_generator_checkpoint,
    prepare_single_prompt_inputs,
    save_video,
)
from pipeline import CausalDiffusionInferencePipeline
from utils.config import normalize_config

# Load config and enable Tri-Attention KV compression

cfg = OmegaConf.load("configs/inference.yaml")
cfg.model.use_tri_attn_kv_compression = True
config = normalize_config(cfg)

# Build the inference pipeline

pipe = CausalDiffusionInferencePipeline(config, device=torch.device("cuda"))
load_generator_checkpoint(pipe.generator, "LongLive-2.0-5B/model_bf16.pt")
pipe = pipe.to(device="cuda", dtype=torch.bfloat16)

# Prepare inputs

prompt = "A futuristic city skyline at sunset."
noise, prompts = prepare_single_prompt_inputs(config, prompt, device="cuda")

# Run inference (KV cache will be compressed internally)

video = pipe.inference(noise=noise, text_prompts=prompts)

# Save the result

save_video(video[0], "output/compressed_kv_demo.mp4", fps=24)

```

The only modification required is the `use_tri_attn_kv_compression` flag; all other pipeline components remain identical to standard inference runs.

## Summary

- **Integrate TriAttention KV cache compression with LongLive** by installing the TriAttention library and setting `use_tri_attn_kv_compression: true` in your configuration.
- The compression mechanism operates in [`wan_5b/modules/causal_model.py`](https://github.com/NVlabs/LongLive/blob/main/wan_5b/modules/causal_model.py) through the `compress_kv_cache` and `decompress_kv_cache` functions imported from the external TriAttention package.
- Memory consumption decreases by approximately 50% while maintaining lossless quality up to the quantization level implemented in [`utils/quant.py`](https://github.com/NVlabs/LongLive/blob/main/utils/quant.py).
- Both single-device inference and sequence-parallel training ([`causal_model_sp_ulysses.py`](https://github.com/NVlabs/LongLive/blob/main/causal_model_sp_ulysses.py)) support the compression toggle.
- The API remains unchanged, ensuring compatibility with existing multi-shot inference and asynchronous decoding workflows.

## Frequently Asked Questions

### How much GPU memory does TriAttention KV cache compression save in LongLive?

According to the source code implementation in [`wan_5b/modules/causal_model.py`](https://github.com/NVlabs/LongLive/blob/main/wan_5b/modules/causal_model.py) and the LongLive changelog, the compression reduces the KV cache memory footprint by approximately **50%**. This is achieved by storing quantized subsets of key and value tensors from the TriAttention library rather than maintaining full-precision representations throughout the generation process.

### Is TriAttention compression lossless or lossy?

The compression is **lossless up to the quantization level** specified by the TriAttention implementation. As implemented in the `_forward_with_cache` method, the `decompress_kv_cache` function restores tensors to their full precision before attention calculations, ensuring no perceptible quality degradation in the generated video output.

### Do I need to modify my existing inference scripts to use KV cache compression?

No. You only need to set the `use_tri_attn_kv_compression` configuration flag to `true` either in your [`configs/inference.yaml`](https://github.com/NVlabs/LongLive/blob/main/configs/inference.yaml) file or programmatically via OmegaConf. The [`utils/config.py`](https://github.com/NVlabs/LongLive/blob/main/utils/config.py) module propagates this setting to the model configuration, and the conditional logic within [`wan_5b/modules/causal_model.py`](https://github.com/NVlabs/LongLive/blob/main/wan_5b/modules/causal_model.py) handles compression and decompression transparently without requiring changes to your inference calling code or pipeline structure.

### Which files in LongLive handle the TriAttention compression logic?

The primary compression logic resides in [`wan_5b/modules/causal_model.py`](https://github.com/NVlabs/LongLive/blob/main/wan_5b/modules/causal_model.py) within the `_update_cache_and_get_kv` method (which applies compression) and the `_forward_with_cache` method (which handles decompression). The sequence-parallel implementation in [`wan_5b/modules/causal_model_sp_ulysses.py`](https://github.com/NVlabs/LongLive/blob/main/wan_5b/modules/causal_model_sp_ulysses.py) contains identical conditional logic for distributed setups, ensuring consistent behavior across different deployment configurations.