How to Debug KV Cache Issues in LingBot-Map Using the LINGBOT_DEBUG_KV Environment Variable

Set LINGBOT_DEBUG_KV to a positive integer to enable per-frame KV cache statistics logging in LingBot-Map's streaming transformers, allowing you to diagnose memory spikes and cache inefficiencies without modifying source code.

The LingBot-Map repository implements transformer-based streaming models that rely on a key-value (KV) cache to maintain attention state across frames. When you encounter unexpected GPU memory growth or performance degradation during long sequences, the LINGBOT_DEBUG_KV environment variable provides real-time visibility into cache behavior according to the actual source code implementation.

How LINGBOT_DEBUG_KV Works Internally

Reading the Environment Variable

In lingbot_map/models/gct_stream.py, the code imports os and reads the variable at module initialization time:

# lingbot_map/models/gct_stream.py

_KV_DEBUG = os.environ.get("LINGBOT_DEBUG_KV", "")

This pattern also appears in lingbot_map/models/gct_stream_window_v2.py (lines 24–25), ensuring both the standard streaming and windowed variants support the same debugging interface.

Parsing the Debug Interval

The implementation parses the string value into an integer interval N that controls print frequency. An empty string disables debugging entirely, while a value of 1 prints every frame and 10 prints every tenth frame. This parsing logic resides in the utility functions of the streaming model files.

Emitting Statistics

During the forward pass, conditional blocks check _KV_DEBUG and emit a formatted table containing the frame index, number of KV entries allocated, cache hit-rate percentage, and memory consumption in MiB. These statistics are printed from within the forward pass of the streaming modules, wrapped by if _KV_DEBUG: guards. The low-level cache implementation in lingbot_map/layers/flashinfer_cache.py interacts with these statistics, though the actual printing occurs in the model files.

When to Enable KV Cache Debugging

Use LINGBOT_DEBUG_KV when you encounter the following scenarios:

• GPU Out-of-Memory errors: Detect if the KV cache grows unbounded during long sequences.
• Sudden inference slowdowns: Spot low hit-rates that force repeated recomputation.
• Verification of model variants: Confirm that new implementations correctly reuse KV entries across windows.
• Memory profiling: Observe how cache footprint scales with different input resolutions.

Enabling and Configuring LINGBOT_DEBUG_KV

Enable debug output by exporting the variable before running your script:

# Print statistics every frame

export LINGBOT_DEBUG_KV=1
python demo.py --input data/example/

For high-frequency inference, reduce verbosity by increasing the interval:

# Print every 20 frames only

export LINGBOT_DEBUG_KV=20
python -m lingbot_map.demo --seq your_sequence.bin

Capture output for later analysis:

export LINGBOT_DEBUG_KV=1
python demo.py 2>&1 | tee kv_debug.log

You can then grep the log for [KV DEBUG] to extract specific statistics.

Interpreting the Debug Output

When enabled, the streaming transformers emit lines formatted as:

[KV DEBUG] frame=23  kv_entries=128  hit_rate=97.7%  mem=84.2MiB
[KV DEBUG] frame=24  kv_entries=128  hit_rate=98.0%  mem=84.2MiB

These lines originate from the forward pass in gct_stream.py and gct_stream_window_v2.py. The hit_rate percentage indicates cache efficiency, while mem shows the current GPU memory footprint. Monitoring these values helps you identify whether the cache implementation in flashinfer_cache.py is functioning correctly or if memory pressure is increasing frame-over-frame.

Summary

• Set LINGBOT_DEBUG_KV to enable non-invasive KV cache monitoring in LingBot-Map.
• The variable accepts integer values representing print intervals; empty values disable output.
• Debug statistics include frame index, entry count, hit-rate, and memory usage.
• Both standard streaming (gct_stream.py) and windowed variants (gct_stream_window_v2.py) support this diagnostic mode.
• Use these metrics to diagnose OOM errors, performance regressions, and cache inefficiencies.

Frequently Asked Questions

What values does LINGBOT_DEBUG_KV accept?

The environment variable accepts positive integers representing the frame interval for printing statistics. A value of 1 outputs every frame, while 10 prints every tenth frame. An empty or unset variable disables debug output entirely, causing the code path to be bypassed with no extra output.

Does enabling LINGBOT_DEBUG_KV affect inference speed?

The debug printing incurs minimal overhead consisting only of string formatting and print calls. It does not modify KV-cache behavior or tensor operations, making it safe for performance-sensitive debugging according to the source implementation.

Where are the debug statistics printed?

Statistics are emitted to stdout during the model's forward pass, specifically from the streaming transformer implementations in lingbot_map/models/gct_stream.py and lingbot_map/models/gct_stream_window_v2.py. You can redirect this output to a log file using standard shell redirection.

Can I use LINGBOT_DEBUG_KV with windowed streaming models?

Yes, the gct_stream_window_v2.py module explicitly implements the same environment variable parsing and statistics emission as the standard streaming model, ensuring consistent debugging across both architectures.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}