# How to Configure keyframe_interval for Optimal KV Cache Memory Usage in LingBot-Map

> Optimize KV cache memory usage in LingBot Map by configuring keyframe_interval. Process thousands of frames on GPU memory without exhaustion.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: performance
- Published: 2026-07-21

---

**Setting `keyframe_interval` greater than 1 in LingBot-Map reduces KV cache memory growth by a factor of N, allowing you to process video sequences containing thousands of frames without exhausting GPU memory.**

The LingBot-Map repository (`Robbyant/lingbot-map`) implements a transformer-based video processing architecture that relies on a KV cache to store attention keys and values for previously processed frames. By default, every frame is retained as a keyframe, causing linear memory growth that quickly exceeds GPU capacity for long videos. Configuring the `keyframe_interval` parameter enables sparse frame retention, dramatically lowering memory consumption while maintaining model inference capabilities across extended temporal windows.

## Understanding KV Cache Memory Constraints

LingBot-Map's architecture stores attention keys and values for frames to enable efficient streaming inference. When `keyframe_interval` is set to its default value of `1`, the model retains KV entries for every single frame processed. This creates linear memory growth relative to sequence length, which rapidly exhausts GPU memory for long videos. Additionally, the RoPE (Rotary Position Embedding) training data only covers approximately 320 frames; beyond this cache size, pose estimation quality begins to degrade. Using a larger `keyframe_interval` allows inference over thousands of frames by limiting the cache to a fixed subset of keyframes, avoiding both memory exhaustion and quality degradation.

## How keyframe_interval Controls Cache Growth

### Default Behavior (Interval = 1)

With the default configuration (`keyframe_interval = 1`), every frame is treated as a keyframe. In [`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py), the `inference_streaming` method determines keyframe status using the following logic:

```python
is_keyframe = (keyframe_interval <= 1) or ((i - scale_frames) % keyframe_interval == 0)

```

When this evaluates to `True`, the model retains the KV tensors for that frame. When `keyframe_interval` is `1`, the modulo condition is bypassed, causing every frame to be stored indefinitely, leading to **O(n)** memory complexity where n equals the number of frames processed.

### Sparse Keyframe Storage (Interval > 1)

Setting `keyframe_interval` to a value **N > 1** modifies the caching strategy to retain only every N-th frame (after the initial scale frames). For non-keyframe indices, the model invokes `_set_skip_append(True)` (as implemented in [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py) lines 68-70), which temporarily disables KV cache appending. This reduces memory growth by roughly a factor of **N**, enabling processing of sequences like `8 + 120 × 13 = 1568` frames within a single window when using `window_size = 128`, `num_scale_frames = 8`, and `keyframe_interval = 13`.

### Flow-Based Keyframe Mode

LingBot-Map also supports an adaptive keyframe selection mechanism controlled by the `flow_threshold` parameter. When you supply a positive `flow_threshold`, the fixed-interval logic is superseded, and frames are automatically promoted to keyframes when optical-flow magnitude exceeds the threshold or when the gap since the last keyframe reaches `max_non_keyframe_gap`. This mode is implemented in the same `inference_streaming` method (see [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py) lines 37-41).

## Configuration Methods

### Command-Line Interface

The recommended approach for benchmarking and evaluation is using the provided scripts. The [`benchmark_gct_memory.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark_gct_memory.py) script parses `--keyframe_interval` at line 64. This allows you to test memory usage across different frame counts:

```bash
python -m scripts.benchmark_gct_memory \
    --device cuda:0 \
    --keyframe_interval 4 \
    --frame_counts 1000 2000 \
    --num_scale_frames 8

```

This command sets the interval to 4, meaning only 25% of frames (after scale frames) are retained in the KV cache.

### Programmatic API

When integrating LingBot-Map into your application, pass `keyframe_interval` directly to the `inference_streaming` method (default value defined at lines 18-19 in [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py)):

```python
from lingbot_map.models.gct_stream_window_v2 import GCTStreamWindowV2

model = GCTStreamWindowV2(...)
outputs = model.inference_streaming(
    images,                # Tensor shape: [B, S, 3, H, W]

    num_scale_frames=8,
    keyframe_interval=5,   # Retain every 5th frame in KV cache

    flow_threshold=0.0,    # Disable flow-based mode, use fixed interval

)

```

### Windowed Inference Considerations

When operating in `--mode windowed`, you typically combine `--keyframe_interval` with `--overlap_keyframes` to ensure sufficient temporal continuity between processing windows. The overlap is automatically scaled by the interval, ensuring at least one paired keyframe exists per window to maintain pose consistency across boundaries (as documented in [`README.md`](https://github.com/Robbyant/lingbot-map/blob/main/README.md) sections 396-399).

## Summary

- **Default risk**: `keyframe_interval = 1` stores every frame, causing rapid GPU memory exhaustion and potential quality degradation beyond ~320 frames due to RoPE limitations.
- **Memory reduction**: Setting `keyframe_interval = N` reduces KV cache growth by factor **N**, calculated as `(i - scale_frames) % keyframe_interval == 0` in [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py).
- **Implementation**: Non-keyframes trigger `_set_skip_append(True)` to prevent cache append operations (lines 68-70).
- **Alternative mode**: Flow-based detection via `flow_threshold` supersedes fixed intervals when enabled.
- **Configuration**: Use `--keyframe_interval` in CLI scripts (e.g., [`benchmark_gct_memory.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark_gct_memory.py) line 64) or pass to `inference_streaming` programmatically.

## Frequently Asked Questions

### What is the default keyframe_interval value in LingBot-Map?

The default value is `1`, which causes the model to treat every processed frame as a keyframe and store its KV tensors in the cache. This default is defined in the `inference_streaming` method signature in [`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py) (lines 18-19).

### How does increasing keyframe_interval affect memory usage exactly?

Setting `keyframe_interval = N` reduces KV cache memory consumption by approximately a factor of **N**. The cache only stores every N-th frame after the initial scale frames, while non-keyframe frames generate predictions without appending to the cache. This transforms memory growth from linear to sub-linear, enabling processing of sequences containing thousands of frames.

### Can flow-based keyframe detection work alongside a fixed keyframe_interval?

When `flow_threshold` is set to a positive value, it supersedes the fixed-interval logic. However, the `max_non_keyframe_gap` parameter ensures that even without sufficient optical flow, a keyframe is forced at regular intervals to prevent excessive gaps. In practice, you either rely on the fixed interval (with `flow_threshold = 0.0`) or the adaptive flow mechanism, not both simultaneously, as the flow mode takes precedence in the implementation logic.

### Why does the cache degrade quality after 320 frames even with sufficient GPU memory?

The degradation occurs because the model's RoPE (Rotary Position Embedding) training data only covers approximately 320 frames. Beyond this length, the positional encodings fall outside the training distribution, causing pose estimation quality to degrade regardless of available memory. Using `keyframe_interval > 1` keeps the effective cache size below this threshold, allowing inference over longer sequences while maintaining quality.