# How to Configure the Keyframe Interval for Managing KV Cache Memory in LingBot-Map

> Learn to configure keyframe interval in LingBot-Map to manage KV cache memory. Reduce GPU memory by setting keyframe_interval > 1 in inference_streaming or via CLI.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-01

---

**Set `keyframe_interval` to a value greater than 1 in `inference_streaming()` or via the `--keyframe_interval` CLI flag to retain only every N-th frame in the transformer KV cache, reducing GPU memory usage by roughly a factor of N.**

LingBot-Map implements a transformer-style KV cache to store attention keys and values for processed video frames. By default, every frame is treated as a keyframe (`keyframe_interval = 1`), which causes the cache to grow linearly and quickly exhaust GPU memory during long video sequences. Configuring the keyframe interval allows you to process thousands of frames while keeping memory consumption bounded.

## Understanding the Keyframe Interval Mechanism

### Default Behavior (Interval = 1)

When `keyframe_interval` is set to `1` (the default), LingBot-Map stores every processed frame in the KV cache. While this maximizes temporal coherence for pose estimation, it creates a memory bottleneck for long videos. The cache size grows proportionally with sequence length, and because the RoPE (Rotary Position Embedding) training data only covers approximately 320 frames, quality degrades beyond this point.

### Memory Reduction with N > 1

Setting `keyframe_interval` to any integer `N > 1` changes the caching strategy to retain only every N-th frame. Non-keyframe frames still generate full predictions, but their KV entries are immediately discarded. This reduces cache growth by roughly a factor of N, enabling inference over sequences containing thousands of frames without exceeding GPU memory limits.

The interval is applied **after** the initial scale frames (the first `num_scale_frames` that are processed together). According to the source code in [`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py), the keyframe decision logic occurs at lines 64-66:

```python
is_keyframe = (keyframe_interval <= 1) or ((i - scale_frames) % keyframe_interval == 0)

```

When `is_keyframe` evaluates to `False`, the model invokes `_set_skip_append(True)` at lines 68-70 to temporarily disable KV cache appending, keeping the cache size constant for that frame.

## Configuring the Keyframe Interval

### Command-Line Interface (CLI)

The recommended approach for benchmarking or batch processing is using the provided scripts. The argument is parsed in `parse_args` within [`scripts/benchmark_gct_memory.py`](https://github.com/Robbyant/lingbot-map/blob/main/scripts/benchmark_gct_memory.py) (around line 64).

```bash
python -m scripts.benchmark_gct_memory \
    --device cuda:0 \
    --keyframe_interval 4 \
    --frame_counts 1000 2000 \
    --num_scale_frames 8

```

This configuration processes the initial 8 scale frames, then retains only every 4th subsequent frame in the KV cache.

### Programmatic API

When integrating LingBot-Map into your own pipeline, pass `keyframe_interval` directly to the `inference_streaming` method in `GCTStreamWindowV2`. The default value is defined at lines 18-19 of [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py).

```python
from lingbot_map.models.gct_stream_window_v2 import GCTStreamWindowV2

model = GCTStreamWindowV2(...)

outputs = model.inference_streaming(
    images,                # Tensor shape: [B, S, 3, H, W]

    num_scale_frames=8,
    keyframe_interval=5,   # Retain every 5th frame in KV cache

    flow_threshold=0.0,    # Disable flow-based keyframe selection

)

```

### Windowed Inference Mode

For extremely long videos, combine `--keyframe_interval` with `--mode windowed`. In this configuration, you typically also specify `--overlap_keyframes` to ensure sufficient temporal overlap between windows. The overlap is automatically scaled by the interval, ensuring at least one paired keyframe exists per window boundary.

With `window_size = 128`, `num_scale_frames = 8`, and `keyframe_interval = 13`, a single window effectively covers `8 + 120 × 13 = 1568` actual video frames while maintaining a fixed KV cache size of 128 entries.

## Flow-Based Keyframe Mode (Alternative)

Beyond fixed intervals, LingBot-Map supports adaptive keyframe selection via optical flow analysis. Supply a positive `flow_threshold` to enable this mode, which supersedes the fixed interval setting.

In flow-based mode (implemented in [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py) at lines 37-41), frames automatically become keyframes when:
- The optical flow magnitude exceeds `flow_threshold`, or
- The gap since the last keyframe reaches `max_non_keyframe_gap`

This dynamic approach preserves detail during rapid motion while compressing static segments.

## Why Keyframe Interval Matters for Long Videos

KV cache memory management directly impacts inference capabilities in three critical ways:

1. **Memory Scaling**: The cache grows linearly with the number of stored keyframes. An interval of 10 reduces memory usage by approximately 90% compared to the default.
2. **Pose Quality Boundaries**: The RoPE embeddings are trained on sequences up to ~320 frames. Beyond this length without keyframe subsampling, positional encoding degrades and pose estimation accuracy drops.
3. **Temporal Range**: Windowed inference with aggressive keyframe intervals (e.g., 13-20) allows processing hour-long videos by treating the KV cache as a sliding window over sparse keyframes rather than dense frame history.

## Summary

- **Default risk**: `keyframe_interval = 1` stores every frame, exhausting GPU memory on long sequences and exceeding the ~320-frame RoPE training limit.
- **Core solution**: Set `keyframe_interval > 1` to retain only every N-th frame after the initial scale frames, reducing memory usage by factor N.
- **Implementation**: Configure via `--keyframe_interval` CLI argument in [`benchmark_gct_memory.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark_gct_memory.py) or the `keyframe_interval` parameter in `inference_streaming()`.
- **Algorithm**: Keyframe status is determined by `(i - scale_frames) % keyframe_interval == 0` in [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py).
- **Advanced option**: Use `flow_threshold` for adaptive keyframe selection based on motion magnitude.
- **Windowed mode**: Combine with `--overlap_keyframes` to process effectively unlimited sequence lengths while maintaining cache bounds.

## Frequently Asked Questions

### What happens to non-keyframe predictions if their KV entries are discarded?

Non-keyframe frames still generate full pose predictions during the forward pass. The model computes attention normally but calls `_set_skip_append(True)` to prevent writing those specific key/value tensors to the persistent cache. The prediction quality remains high for individual frames, though temporal consistency depends on the density of retained keyframes.

### How do I choose the optimal keyframe interval for my video?

Select an interval based on your GPU memory capacity and motion characteristics. For static scenes with slow movement, values between 10-20 work well. For high-motion content, use 3-5 or switch to flow-based mode with `flow_threshold=0.5` and `max_non_keyframe_gap=30`. Always ensure your interval keeps the effective cache size under the ~320-frame RoPE limit mentioned in the README.

### Can I use different keyframe intervals for different video segments?

The current implementation in `GCTStreamWindowV2` uses a global `keyframe_interval` parameter for the entire sequence. To achieve variable intervals, you would need to manually implement frame selection logic before calling `inference_streaming()` with custom keyframe masks, or process the video in chunks with different interval settings and merge results.

### Does increasing the keyframe interval affect inference speed?

Yes, but favorably. With fewer keyframes, the KV cache remains smaller, reducing memory bandwidth pressure during attention computation. However, the primary benefit is enabling longer effective context windows; the per-frame computation cost remains constant since non-keyframes still require a full forward pass to generate predictions.