How to Use the LINGBOT_DEBUG_KV Environment Variable for KV Cache Debugging in LingBot-Map

Set the LINGBOT_DEBUG_KV environment variable to a positive integer to enable per-frame KV cache statistics printed from the streaming transformer models.

LingBot-Map is a transformer-based mapping framework that utilizes attention key/value (KV) caching to optimize sequential processing. When debugging memory issues or cache inefficiencies in the streaming models, the LINGBOT_DEBUG_KV environment variable provides real-time visibility into cache allocation, hit rates, and memory pressure without modifying model behavior.

How LINGBOT_DEBUG_KV Works Internally

The debugging mechanism operates through three distinct phases in the streaming model source code: reading the environment variable, parsing the interval, and conditionally printing statistics.

Reading the Variable

Both gct_stream.py and gct_stream_window_v2.py import the os module and read LINGBOT_DEBUG_KV at import time. In lingbot_map/models/gct_stream.py (lines 24-25), the code retrieves the variable:

_KV_DEBUG = os.environ.get("LINGBOT_DEBUG_KV", "")

If the variable is unset or empty, the debugging pipeline remains inactive with zero runtime overhead.

Parsing the Interval

A utility function in lingbot_map/models/gct_stream_window_v2.py (lines 35-36) parses the string value into an integer interval N. When LINGBOT_DEBUG_KV is set to a number (e.g., 1 or 10), the system prints statistics every N frames. An empty string disables output entirely, while 1 provides frame-by-frame logging.

Printing Statistics

Inside the forward pass of the streaming modules, conditional blocks check _KV_DEBUG and emit a formatted table containing:

• Frame index: Current position in the sequence
• KV entries: Number of allocated cache entries
• Cache hit-rate: Percentage of cache reuses
• Memory used: MiB consumed by the cache

The print statements are wrapped in if _KV_DEBUG: guards to ensure they execute only when debugging is enabled. This design incurs minimal overhead—only string formatting and a print call—without altering the underlying KV-cache logic.

When to Enable KV Debugging

Enable LINGBOT_DEBUG_KV when investigating specific runtime behaviors:

• Unexpected GPU OOM errors: Detect if the KV cache grows unbounded during long sequences
• Sudden performance degradation: Identify low hit-rates causing repeated recomputation
• Cache reuse verification: Confirm that new model variants correctly reuse KV entries across windows
• Memory profiling: Observe memory scaling across different input resolutions

How to Enable KV Debugging

Set the environment variable before launching your Python script. The value determines the print interval.

Basic Usage (Every Frame)

export LINGBOT_DEBUG_KV=1
python demo.py --input data/example/loop/

Custom Interval (Every 10 Frames)

export LINGBOT_DEBUG_KV=10
python -m lingbot_map.demo --seq your_sequence.bin

Capturing Output to File

export LINGBOT_DEBUG_KV=1
python demo.py 2>&1 | tee kv_debug.log

After capture, grep the log for [KV DEBUG] to extract statistics:

grep "\[KV DEBUG\]" kv_debug.log > stats_only.log

Understanding the Debug Output

When enabled, the streaming transformer emits lines formatted as:

[KV DEBUG] frame=23  kv_entries=128  hit_rate=97.7%  mem=84.2MiB
[KV DEBUG] frame=24  kv_entries=128  hit_rate=98.0%  mem=84.2MiB

These metrics originate from the forward pass in gct_stream.py and gct_stream_window_v2.py. Monitor the kv_entries column for unbounded growth and the hit_rate column to verify cache efficiency. Stable mem values across frames indicate proper cache management, while monotonic increases suggest memory leaks.

Key Source Files

The KV debugging implementation spans three critical files in the repository:

• lingbot_map/models/gct_stream.py: Main streaming transformer containing the environment variable hook and primary printing logic
• lingbot_map/models/gct_stream_window_v2.py: Windowed streaming variant with interval parsing and conditional statistics output
• lingbot_map/layers/flashinfer_cache.py: Low-level KV-cache implementation that provides the raw statistics consumed by the debug output

Summary

• Set LINGBOT_DEBUG_KV to a positive integer (e.g., 1, 10, 20) to enable per-frame KV cache statistics in LingBot-Map
• Leave it unset or empty to disable debugging and eliminate runtime overhead
• Monitor output for kv_entries, hit_rate, and mem values to diagnose memory spikes and cache inefficiencies
• Use file redirection to capture logs for post-hoc analysis of long-running sequences
• Minimal performance impact: Debug printing only adds string formatting overhead without changing model behavior

Frequently Asked Questions

What values can I set for LINGBOT_DEBUG_KV?

Set LINGBOT_DEBUG_KV to any positive integer to specify the print interval. A value of 1 prints statistics every frame, while 10 prints every 10th frame. An empty string or unset variable disables output entirely. According to the source in gct_stream_window_v2.py, the parser converts the string to an integer that controls the sampling frequency.

Does enabling LINGBOT_DEBUG_KV slow down inference?

The debugging mechanism adds negligible overhead. The code only performs string formatting and print calls when the variable is set, and it does not modify the KV-cache operations or model weights. Performance impact is limited to the I/O cost of printing to stdout, which is typically insignificant compared to transformer computation.

How do I distinguish between the streaming model variants when debugging?

gct_stream.py implements the base streaming transformer, while gct_stream_window_v2.py handles windowed streaming with vocabulary-aware caching. Both respect LINGBOT_DEBUG_KV, but gct_stream_window_v2.py includes additional logic for parsing the interval. If you see window-specific statistics in the output, the model is likely using the windowed variant.

Can I use LINGBOT_DEBUG_KV in production environments?

While technically possible, avoid setting LINGBOT_DEBUG_KV in production due to stdout pollution and potential log file bloat. The variable is designed for development and profiling scenarios. For production monitoring, instrument the flashinfer_cache.py layer directly with structured logging rather than relying on printed debug statements.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}