How to Configure keyframe_interval for Optimal KV Cache Memory Usage in LingBot-Map

Setting keyframe_interval greater than 1 in LingBot-Map reduces KV cache memory growth by a factor of N, allowing you to process video sequences containing thousands of frames without exhausting GPU memory.

The LingBot-Map repository (Robbyant/lingbot-map) implements a transformer-based video processing architecture that relies on a KV cache to store attention keys and values for previously processed frames. By default, every frame is retained as a keyframe, causing linear memory growth that quickly exceeds GPU capacity for long videos. Configuring the keyframe_interval parameter enables sparse frame retention, dramatically lowering memory consumption while maintaining model inference capabilities across extended temporal windows.

Understanding KV Cache Memory Constraints

LingBot-Map's architecture stores attention keys and values for frames to enable efficient streaming inference. When keyframe_interval is set to its default value of 1, the model retains KV entries for every single frame processed. This creates linear memory growth relative to sequence length, which rapidly exhausts GPU memory for long videos. Additionally, the RoPE (Rotary Position Embedding) training data only covers approximately 320 frames; beyond this cache size, pose estimation quality begins to degrade. Using a larger keyframe_interval allows inference over thousands of frames by limiting the cache to a fixed subset of keyframes, avoiding both memory exhaustion and quality degradation.

How keyframe_interval Controls Cache Growth

Default Behavior (Interval = 1)

With the default configuration (keyframe_interval = 1), every frame is treated as a keyframe. In lingbot_map/models/gct_stream_window_v2.py, the inference_streaming method determines keyframe status using the following logic:

is_keyframe = (keyframe_interval <= 1) or ((i - scale_frames) % keyframe_interval == 0)

When this evaluates to True, the model retains the KV tensors for that frame. When keyframe_interval is 1, the modulo condition is bypassed, causing every frame to be stored indefinitely, leading to O(n) memory complexity where n equals the number of frames processed.

Sparse Keyframe Storage (Interval > 1)

Setting keyframe_interval to a value N > 1 modifies the caching strategy to retain only every N-th frame (after the initial scale frames). For non-keyframe indices, the model invokes _set_skip_append(True) (as implemented in gct_stream_window_v2.py lines 68-70), which temporarily disables KV cache appending. This reduces memory growth by roughly a factor of N, enabling processing of sequences like 8 + 120 × 13 = 1568 frames within a single window when using window_size = 128, num_scale_frames = 8, and keyframe_interval = 13.

Flow-Based Keyframe Mode

LingBot-Map also supports an adaptive keyframe selection mechanism controlled by the flow_threshold parameter. When you supply a positive flow_threshold, the fixed-interval logic is superseded, and frames are automatically promoted to keyframes when optical-flow magnitude exceeds the threshold or when the gap since the last keyframe reaches max_non_keyframe_gap. This mode is implemented in the same inference_streaming method (see gct_stream_window_v2.py lines 37-41).

Configuration Methods

Command-Line Interface

The recommended approach for benchmarking and evaluation is using the provided scripts. The benchmark_gct_memory.py script parses --keyframe_interval at line 64. This allows you to test memory usage across different frame counts:

python -m scripts.benchmark_gct_memory \
    --device cuda:0 \
    --keyframe_interval 4 \
    --frame_counts 1000 2000 \
    --num_scale_frames 8

This command sets the interval to 4, meaning only 25% of frames (after scale frames) are retained in the KV cache.

Programmatic API

When integrating LingBot-Map into your application, pass keyframe_interval directly to the inference_streaming method (default value defined at lines 18-19 in gct_stream_window_v2.py):

from lingbot_map.models.gct_stream_window_v2 import GCTStreamWindowV2

model = GCTStreamWindowV2(...)
outputs = model.inference_streaming(
    images,                # Tensor shape: [B, S, 3, H, W]

    num_scale_frames=8,
    keyframe_interval=5,   # Retain every 5th frame in KV cache

    flow_threshold=0.0,    # Disable flow-based mode, use fixed interval

)

Windowed Inference Considerations

When operating in --mode windowed, you typically combine --keyframe_interval with --overlap_keyframes to ensure sufficient temporal continuity between processing windows. The overlap is automatically scaled by the interval, ensuring at least one paired keyframe exists per window to maintain pose consistency across boundaries (as documented in README.md sections 396-399).

Summary

• Default risk: keyframe_interval = 1 stores every frame, causing rapid GPU memory exhaustion and potential quality degradation beyond ~320 frames due to RoPE limitations.
• Memory reduction: Setting keyframe_interval = N reduces KV cache growth by factor N, calculated as (i - scale_frames) % keyframe_interval == 0 in gct_stream_window_v2.py.
• Implementation: Non-keyframes trigger _set_skip_append(True) to prevent cache append operations (lines 68-70).
• Alternative mode: Flow-based detection via flow_threshold supersedes fixed intervals when enabled.
• Configuration: Use --keyframe_interval in CLI scripts (e.g., benchmark_gct_memory.py line 64) or pass to inference_streaming programmatically.

Frequently Asked Questions

What is the default keyframe_interval value in LingBot-Map?

The default value is 1, which causes the model to treat every processed frame as a keyframe and store its KV tensors in the cache. This default is defined in the inference_streaming method signature in lingbot_map/models/gct_stream_window_v2.py (lines 18-19).

How does increasing keyframe_interval affect memory usage exactly?

Setting keyframe_interval = N reduces KV cache memory consumption by approximately a factor of N. The cache only stores every N-th frame after the initial scale frames, while non-keyframe frames generate predictions without appending to the cache. This transforms memory growth from linear to sub-linear, enabling processing of sequences containing thousands of frames.

Can flow-based keyframe detection work alongside a fixed keyframe_interval?

When flow_threshold is set to a positive value, it supersedes the fixed-interval logic. However, the max_non_keyframe_gap parameter ensures that even without sufficient optical flow, a keyframe is forced at regular intervals to prevent excessive gaps. In practice, you either rely on the fixed interval (with flow_threshold = 0.0) or the adaptive flow mechanism, not both simultaneously, as the flow mode takes precedence in the implementation logic.

Why does the cache degrade quality after 320 frames even with sufficient GPU memory?

The degradation occurs because the model's RoPE (Rotary Position Embedding) training data only covers approximately 320 frames. Beyond this length, the positional encodings fall outside the training distribution, causing pose estimation quality to degrade regardless of available memory. Using keyframe_interval > 1 keeps the effective cache size below this threshold, allowing inference over longer sequences while maintaining quality.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}