# How to Debug KV Cache Issues in LingBot-Map Using the LINGBOT_DEBUG_KV Environment Variable

> Debug KV cache issues in LingBot-Map with LINGBOT_DEBUG_KV. Enable per-frame statistics logging to diagnose memory spikes and cache inefficiencies without code changes.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-06-29

---

**Set `LINGBOT_DEBUG_KV` to a positive integer to enable per-frame KV cache statistics logging in LingBot-Map's streaming transformers, allowing you to diagnose memory spikes and cache inefficiencies without modifying source code.**

The LingBot-Map repository implements transformer-based streaming models that rely on a key-value (KV) cache to maintain attention state across frames. When you encounter unexpected GPU memory growth or performance degradation during long sequences, the `LINGBOT_DEBUG_KV` environment variable provides real-time visibility into cache behavior according to the actual source code implementation.

## How LINGBOT_DEBUG_KV Works Internally

### Reading the Environment Variable

In [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py), the code imports `os` and reads the variable at module initialization time:

```python

# lingbot_map/models/gct_stream.py

_KV_DEBUG = os.environ.get("LINGBOT_DEBUG_KV", "")

```

This pattern also appears in [`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py) (lines 24–25), ensuring both the standard streaming and windowed variants support the same debugging interface.

### Parsing the Debug Interval

The implementation parses the string value into an integer interval *N* that controls print frequency. An empty string disables debugging entirely, while a value of `1` prints every frame and `10` prints every tenth frame. This parsing logic resides in the utility functions of the streaming model files.

### Emitting Statistics

During the forward pass, conditional blocks check `_KV_DEBUG` and emit a formatted table containing the **frame index**, **number of KV entries allocated**, **cache hit-rate percentage**, and **memory consumption in MiB**. These statistics are printed from within the forward pass of the streaming modules, wrapped by `if _KV_DEBUG:` guards. The low-level cache implementation in [`lingbot_map/layers/flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/flashinfer_cache.py) interacts with these statistics, though the actual printing occurs in the model files.

## When to Enable KV Cache Debugging

Use `LINGBOT_DEBUG_KV` when you encounter the following scenarios:

- **GPU Out-of-Memory errors**: Detect if the KV cache grows unbounded during long sequences.
- **Sudden inference slowdowns**: Spot low hit-rates that force repeated recomputation.
- **Verification of model variants**: Confirm that new implementations correctly reuse KV entries across windows.
- **Memory profiling**: Observe how cache footprint scales with different input resolutions.

## Enabling and Configuring LINGBOT_DEBUG_KV

Enable debug output by exporting the variable before running your script:

```bash

# Print statistics every frame

export LINGBOT_DEBUG_KV=1
python demo.py --input data/example/

```

For high-frequency inference, reduce verbosity by increasing the interval:

```bash

# Print every 20 frames only

export LINGBOT_DEBUG_KV=20
python -m lingbot_map.demo --seq your_sequence.bin

```

Capture output for later analysis:

```bash
export LINGBOT_DEBUG_KV=1
python demo.py 2>&1 | tee kv_debug.log

```

You can then grep the log for `[KV DEBUG]` to extract specific statistics.

## Interpreting the Debug Output

When enabled, the streaming transformers emit lines formatted as:

```

[KV DEBUG] frame=23  kv_entries=128  hit_rate=97.7%  mem=84.2MiB
[KV DEBUG] frame=24  kv_entries=128  hit_rate=98.0%  mem=84.2MiB

```

These lines originate from the forward pass in [`gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream.py) and [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py). The **hit_rate** percentage indicates cache efficiency, while **mem** shows the current GPU memory footprint. Monitoring these values helps you identify whether the cache implementation in [`flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/flashinfer_cache.py) is functioning correctly or if memory pressure is increasing frame-over-frame.

## Summary

- Set `LINGBOT_DEBUG_KV` to enable non-invasive KV cache monitoring in LingBot-Map.
- The variable accepts integer values representing print intervals; empty values disable output.
- Debug statistics include frame index, entry count, hit-rate, and memory usage.
- Both standard streaming ([`gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream.py)) and windowed variants ([`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py)) support this diagnostic mode.
- Use these metrics to diagnose OOM errors, performance regressions, and cache inefficiencies.

## Frequently Asked Questions

### What values does LINGBOT_DEBUG_KV accept?

The environment variable accepts positive integers representing the frame interval for printing statistics. A value of `1` outputs every frame, while `10` prints every tenth frame. An empty or unset variable disables debug output entirely, causing the code path to be bypassed with no extra output.

### Does enabling LINGBOT_DEBUG_KV affect inference speed?

The debug printing incurs minimal overhead consisting only of string formatting and `print` calls. It does not modify KV-cache behavior or tensor operations, making it safe for performance-sensitive debugging according to the source implementation.

### Where are the debug statistics printed?

Statistics are emitted to stdout during the model's forward pass, specifically from the streaming transformer implementations in [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py) and [`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py). You can redirect this output to a log file using standard shell redirection.

### Can I use LINGBOT_DEBUG_KV with windowed streaming models?

Yes, the [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py) module explicitly implements the same environment variable parsing and statistics emission as the standard streaming model, ensuring consistent debugging across both architectures.