Default Values for KV Cache Parameters in LingBot-Map: Complete Configuration Guide

The default values for LingBot-Map's KV cache parameters are a 64-frame sliding window, 8 permanent scale frames, with special token retention enabled, camera-only mode disabled, and scale frame inclusion enabled.

LingBot-Map utilizes a configurable key-value (KV) cache to maintain attention states across video frames for real-time SLAM and scene understanding. As implemented in the Robbyant/lingbot-map repository, these caching behaviors are controlled through specific hyper-parameters defined in the model constructor, with sensible defaults that balance memory consumption against temporal context retention.

KV Cache Parameter Defaults in LingBot-Map

According to the constructor signature and docstring in lingbot_map/models/gct_stream_window_v2.py (lines 223-260), the system initializes five primary parameters that govern cache eviction and retention policies:

• kv_cache_sliding_window: 64 (int) — The maximum number of recent frames retained in the cache before FIFO eviction occurs.
• kv_cache_scale_frames: 8 (int) — The number of scale frames permanently preserved regardless of sliding window limits.
• kv_cache_cross_frame_special: True (bool) — When enabled, retains special tokens (e.g., CLS tokens) from evicted frames to maintain cross-frame attention capabilities.
• kv_cache_include_scale_frames: True (bool) — Determines whether scale frames are additionally stored within the KV cache itself.
• kv_cache_camera_only: False (bool) — Restricts cache retention to camera-related tokens only when set to True, significantly reducing memory overhead for non-camera data.

Where These Defaults Are Defined

GCTStreamWindowV2 Implementation

The primary definition resides in lingbot_map/models/gct_stream_window_v2.py within the class constructor between lines 223-260. This implementation establishes the default 64-frame sliding window and 8-frame scale retention policy, along with the helper method clean_kv_cache() for explicit cache purging.

Legacy Stream Window Support

An identical parameter interface exists in lingbot_map/models/gct_stream_window.py (lines 160-197), ensuring backward compatibility for earlier stream-window variants. Both files maintain consistent default values and expose the same internal flags (_skip_append, _defer_eviction) that toggle during inference.

How to Override Default Values

You can customize these parameters during model instantiation to optimize for specific hardware constraints or temporal context requirements:

from lingbot_map.models.gct_stream_window_v2 import GCTStreamWindowV2

# Configure a tighter memory budget

model = GCTStreamWindowV2(
    kv_cache_sliding_window=32,          # Default: 64

    kv_cache_scale_frames=4,             # Default: 8

    kv_cache_cross_frame_special=True,   # Default: True

    kv_cache_include_scale_frames=True,  # Default: True

    kv_cache_camera_only=False,          # Default: False

)

For dynamic adjustments during inference without reinstantiation, modify the attributes directly:

def adjust_cache_parameters(model, window, scale):
    model.kv_cache_sliding_window = window
    model.kv_cache_scale_frames = scale

# Reduce cache size after initialization

adjust_cache_parameters(model, window=16, scale=2)

Low-Level Cache Implementation

The configured parameters interface with the underlying cache mechanism in lingbot_map/layers/flashinfer_cache.py, which handles the actual memory management operations. The model's clean_kv_cache() method provides explicit control to purge the cache state, while internal flags including _skip_append and _defer_eviction automatically adjust cache behavior during forward passes based on the configured default values.

Summary

• Default sliding window: 64 frames (kv_cache_sliding_window=64)
• Default scale frames: 8 frames (kv_cache_scale_frames=8)
• Special token retention: Enabled by default (kv_cache_cross_frame_special=True)
• Scale frame storage: Enabled by default (kv_cache_include_scale_frames=True)
• Camera-only mode: Disabled by default (kv_cache_camera_only=False)
• Primary source: lingbot_map/models/gct_stream_window_v2.py (lines 223-260)
• Legacy support: lingbot_map/models/gct_stream_window.py (lines 160-197)

Frequently Asked Questions

What is the default sliding window size for the KV cache in LingBot-Map?

The default sliding window size is 64 frames, as defined by the kv_cache_sliding_window parameter in lingbot_map/models/gct_stream_window_v2.py. When the cache exceeds this limit, the oldest frames are evicted to maintain memory constraints while preserving the 64 most recent attention states.

How do I completely disable the sliding window eviction in LingBot-Map?

While the default value is 64, you can effectively disable eviction by setting kv_cache_sliding_window to an integer larger than your total video frame count. However, memory usage will grow linearly with sequence length unless you also adjust kv_cache_scale_frames and enable kv_cache_camera_only to limit token retention.

What is the difference between scale frames and sliding window frames?

Scale frames (default: 8) are permanently retained in memory regardless of the sliding window eviction policy, providing stable anchor points for multi-scale attention mechanisms. Sliding window frames (default: 64) are subject to FIFO eviction when exceeding capacity. The kv_cache_include_scale_frames parameter controls whether these permanent scale frames are additionally stored within the KV cache structure itself or maintained separately.

Where can I benchmark different KV cache configurations?

The repository provides scripts/benchmark_gct_memory.py (lines 152-155), which demonstrates command-line parameter passing for performance testing. This script allows you to measure memory consumption and inference latency across various combinations of kv_cache_sliding_window and kv_cache_scale_frames values to find optimal settings for your hardware.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}