How to Configure keyframe_interval for Sequences Exceeding 320 Frames in LingBot-Map

Set keyframe_interval to an integer greater than 1 (e.g., 8 or 10) in the inference_streaming() method or via the --keyframe-interval CLI flag to subsample the KV-cache and prevent GPU out-of-memory errors when processing videos longer than 320 frames.

LingBot-Map processes video streams using a keyframe-based KV-cache that, by default, stores every single frame (keyframe_interval=1). When handling sequences exceeding approximately 320 frames, this linear memory growth quickly exhausts GPU resources. Configuring the keyframe_interval parameter allows you to retain only every n-th frame in cache, enabling inference on arbitrarily long videos without memory errors according to the Robbyant/lingbot-map source code.

Understanding the Default Memory Limitations

In lingbot_map/models/gct_stream_window_v2.py, the streaming inference implementation treats every frame as a keyframe by default. According to the source code (lines 30–36), this keyframe_interval=1 behavior causes the KV-cache to grow linearly with sequence length. For videos longer than roughly 320 frames, this growth exceeds the memory capacity of most GPUs, triggering out-of-memory errors.

Configuring Fixed-Interval Keyframes for Long Sequences

The most reliable way to handle long sequences is to set a fixed keyframe_interval greater than 1. This configuration retains only every n-th frame (after the initial scale frames) in the KV-cache, reducing memory usage by roughly a factor of n.

Python API Implementation

Pass the keyframe_interval argument directly to the inference_streaming() method defined in lingbot_map/models/gct_stream_window_v2.py (lines 18–49). Set flow_threshold=0.0 to ensure the fixed-interval mode is active:

import torch
from lingbot_map.models.gct_stream_window_v2 import GCTModelV2

model = GCTModelV2(...)
frames = torch.randn(1, 500, 3, 480, 640)   # 500 frames, exceeds 320 limit

# Store every 8th frame to reduce KV-cache by factor of 8

output = model.inference_streaming(
    images=frames,
    keyframe_interval=8,          # Configure keyframe subsampling

    flow_threshold=0.0,          # Disable flow-based override

    max_non_keyframe_gap=30,
)

Command-Line Configuration

When using the provided benchmark script, specify the interval via the --keyframe-interval flag defined in scripts/benchmark_gct_memory.py at line 64:

python scripts/benchmark_gct_memory.py \
    --data /path/to/frames \
    --keyframe-interval 10   # Retain every 10th frame

Alternative: Flow-Based Keyframe Detection

As an alternative to fixed intervals, enable dynamic keyframe selection by setting flow_threshold greater than 0.0. According to lines 37–41 in lingbot_map/models/gct_stream_window_v2.py, this mode automatically selects keyframes based on optical-flow magnitude and overrides the keyframe_interval setting. Use this when scene changes are sparse and irregular, though for predictable long-sequence processing the fixed-interval method provides more deterministic memory management.

Choosing the Right Interval for Your GPU

Select a keyframe_interval that fits your GPU memory budget. For sequences of 500 frames, a value of 8–10 typically prevents OOM errors while preserving sufficient temporal context. For sequences exceeding 1000 frames, consider 16–20. The memory reduction scales linearly: an interval of 10 reduces cache size to approximately 10% of the default full-frame cache.

Summary

• Default behavior (keyframe_interval=1) stores every frame and causes GPU out-of-memory errors on sequences longer than ~320 frames.
• Fixed-interval configuration reduces KV-cache memory consumption by a factor equal to the keyframe_interval value.
• Implementation is available via the keyframe_interval parameter in inference_streaming() or the --keyframe-interval CLI flag in scripts/benchmark_gct_memory.py.
• Flow-based mode (flow_threshold > 0) provides automatic keyframe selection but overrides fixed intervals.

Frequently Asked Questions

What is the default keyframe_interval in LingBot-Map?

The default value is 1, meaning every frame is treated as a keyframe and stored in the KV-cache. This behavior is implemented in lingbot_map/models/gct_stream_window_v2.py (lines 30–36).

How does increasing keyframe_interval affect memory usage?

Memory consumption decreases by roughly a factor equal to the keyframe_interval value. For example, setting keyframe_interval=10 reduces the KV-cache size to approximately 10% of the default full-frame cache, allowing sequences well beyond 320 frames to fit within standard GPU memory limits.

Can I use both keyframe_interval and flow_threshold simultaneously?

No. When flow_threshold is set greater than 0.0, the flow-based detection mode automatically overrides the fixed keyframe_interval and selects keyframes dynamically based on optical-flow magnitude, as defined in lines 37–41 of the source file.

Where is the --keyframe-interval CLI argument defined?

The command-line flag is defined in scripts/benchmark_gct_memory.py at line 64, which forwards the integer value to the underlying model's streaming inference method.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}