How the keyframe_interval Parameter Impacts KV Cache Memory and Reconstruction Quality

The keyframe_interval parameter controls which frames store their Key-Value tensors in the KV cache during streaming inference, reducing memory growth by a factor of 1/N when set to N, while maintaining full per-frame prediction accuracy with only minor trade-offs in long-range temporal consistency.

In the Robbyant/lingbot-map repository, keyframe_interval functions as a critical memory optimization lever for streaming vision models. This parameter determines the sampling frequency of frames that persist in the model's KV cache, directly governing GPU memory consumption during long-sequence inference without sacrificing individual frame reconstruction quality.

Understanding keyframe_interval in the Lingbot Architecture

The keyframe_interval parameter operates within the GCTStreamWindowV2 model class to distinguish between keyframes (stored in cache) and non-keyframes (processed but discarded). According to the implementation docstring in lingbot_map/models/gct_stream_window_v2.py (lines 30-36), this parameter sets the stride at which frames are retained in the KV cache after the initial scale frames are processed.

When processing a video stream, the model evaluates each frame's status based on this interval. Keyframes undergo full KV tensor computation and cache insertion, while non-keyframes execute a modified forward pass that avoids permanent cache growth.

KV Cache Memory Impact

Linear Growth with keyframe_interval=1

When keyframe_interval is set to 1, the model treats every frame as a keyframe. In this configuration, the Key-Value tensors for each processed frame are appended to the running cache, causing memory usage to grow linearly with the number of frames. This mode provides maximum temporal context but consumes the most GPU memory, making it suitable for short sequences or hardware with abundant VRAM.

Sub-linear Growth with Sparse Keyframes

For values greater than 1, the implementation stores only every N-th frame in the KV cache, reducing memory growth by approximately a factor of 1 / keyframe_interval. The source code achieves this through the helper method _set_skip_append(True), which is invoked for non-keyframes to bypass the cache-append operation (see lingbot_map/models/gct_stream_window_v2.py, lines 4-22).

The benchmark script scripts/benchmark_gct_memory.py (lines 28-33) demonstrates this behavior by toggling the cache-skip logic based on the current frame index modulo the keyframe interval. This sparse caching strategy enables processing of significantly longer video sequences on constrained hardware.

Reconstruction Quality Implications

Preserved Per-Frame Accuracy

Regardless of keyframe status, every frame produces a full prediction for depth, pose, and other output modalities. For non-keyframes, the model attends to both the cached Key-Value tensors from the most recent keyframe and the current frame's computed KV representation. After the forward pass completes, the current frame's KV tensors are discarded, but the prediction quality remains essentially identical to keyframe processing because the attention mechanism still leverages historical context from the cached keyframe.

Temporal Consistency Trade-offs

The primary quality impact manifests in long-range temporal consistency rather than per-frame accuracy. When keyframe_interval is set to large values, non-keyframes never contribute their KV representations to subsequent frames, potentially creating subtle discontinuities in motion consistency across extended intervals. The default setting of keyframe_interval=1 delivers the highest fidelity for temporal coherence, while larger values trade a modest amount of long-range consistency for substantial memory savings.

Source Code Implementation Details

The cache control logic resides in two critical locations within the repository:

• lingbot_map/models/gct_stream_window_v2.py (lines 4-22): Contains the _set_skip_append() and _set_defer_eviction() helper methods that programmatically enable or disable KV cache appending based on frame type.

• scripts/benchmark_gct_memory.py (lines 28-33): Implements the frame selection logic that determines whether the current index qualifies as a keyframe, triggering the appropriate cache-skip behavior.

The aggregator class in lingbot_map/aggregator/base.py manages the underlying KV cache structure that these methods manipulate, storing the persistent Key-Value tensors across the streaming inference loop.

Configuring keyframe_interval in Practice

To evaluate memory usage across different interval settings, instantiate the streaming model with varying parameters:

from lingbot_map.models.gct_stream_window_v2 import GCTStreamWindowV2
import torch

model = GCTStreamWindowV2(...)
images = torch.randn(1, 50, 3, 480, 640)          # 50 frames

# Maximum memory usage: every frame cached

out_dense = model.inference_streaming(
    images,
    num_scale_frames=5,
    keyframe_interval=1,
    output_device=torch.device('cpu')
)

# 75% memory reduction: only every 4th frame cached

out_sparse = model.inference_streaming(
    images,
    num_scale_frames=5,
    keyframe_interval=4,
    output_device=torch.device('cpu')
)

print(model.get_kv_cache_info())

For systematic benchmarking across sequence lengths, execute the provided benchmark script:

# Baseline: full KV cache accumulation

python scripts/benchmark_gct_memory.py \
    --frame-counts 1000 \
    --keyframe-interval 1

# Optimized: 8x memory reduction

python scripts/benchmark_gct_memory.py \
    --frame-counts 1000 \
    --keyframe-interval 8

The resulting CSV output shows peak GPU memory scaling inversely with the interval value, while inference time remains stable, confirming that keyframe_interval primarily functions as a memory optimization rather than a computational accelerator.

Summary

• keyframe_interval controls KV cache update frequency by determining which frames persist their Key-Value tensors during streaming inference.

• Memory scales inversely with the interval value—setting the parameter to N reduces cache growth by approximately 1/N compared to the default value of 1.

• All frames generate full predictions regardless of keyframe status, ensuring consistent per-frame reconstruction quality.

• Temporal consistency may degrade slightly with very large intervals due to non-keyframes never contributing to future attention contexts.

• Implementation relies on the _set_skip_append() helper in gct_stream_window_v2.py to bypass cache writes for non-keyframes.

Frequently Asked Questions

What is the default keyframe_interval in lingbot-map?

The default value is 1, meaning every processed frame is treated as a keyframe and stored in the KV cache. This configuration provides maximum temporal consistency at the cost of linear memory growth with sequence length.

Does increasing keyframe_interval reduce inference speed?

No, increasing the interval does not significantly reduce inference time per frame. While non-keyframes skip the cache-append operation, they still require full forward pass computation. The primary benefit is reduced GPU memory consumption, enabling longer sequences to fit within hardware constraints.

How does keyframe_interval affect video reconstruction quality?

Per-frame reconstruction quality for depth and pose estimation remains essentially unchanged because every frame computes a full prediction using the most recent cached keyframe's context. However, very large intervals may introduce minor discontinuities in long-range temporal consistency since intermediate frames do not contribute their KV representations to future frames.

Can I change keyframe_interval dynamically during inference?

While the source code in gct_stream_window_v2.py supports parameter updates between inference calls, the interval is typically fixed for the duration of a sequence to maintain consistent cache behavior. Dynamic adjustment would require careful management of the existing cache state to ensure temporal coherence is preserved.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}