# How to Use the LINGBOT_DEBUG_KV Environment Variable for KV Cache Debugging in LingBot-Map

> Debug KV cache in LingBot-Map by setting the LINGBOT_DEBUG_KV environment variable. Enable per-frame statistics from streaming transformer models for deeper insights. Learn how now.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-01

---

**Set the `LINGBOT_DEBUG_KV` environment variable to a positive integer to enable per-frame KV cache statistics printed from the streaming transformer models.**

LingBot-Map is a transformer-based mapping framework that utilizes attention key/value (KV) caching to optimize sequential processing. When debugging memory issues or cache inefficiencies in the streaming models, the **`LINGBOT_DEBUG_KV`** environment variable provides real-time visibility into cache allocation, hit rates, and memory pressure without modifying model behavior.

## How LINGBOT_DEBUG_KV Works Internally

The debugging mechanism operates through three distinct phases in the streaming model source code: reading the environment variable, parsing the interval, and conditionally printing statistics.

### Reading the Variable

Both [`gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream.py) and [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py) import the `os` module and read `LINGBOT_DEBUG_KV` at import time. In [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py) (lines 24-25), the code retrieves the variable:

```python
_KV_DEBUG = os.environ.get("LINGBOT_DEBUG_KV", "")

```

If the variable is unset or empty, the debugging pipeline remains inactive with zero runtime overhead.

### Parsing the Interval

A utility function in [`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py) (lines 35-36) parses the string value into an integer interval *N*. When `LINGBOT_DEBUG_KV` is set to a number (e.g., `1` or `10`), the system prints statistics every *N* frames. An empty string disables output entirely, while `1` provides frame-by-frame logging.

### Printing Statistics

Inside the forward pass of the streaming modules, conditional blocks check `_KV_DEBUG` and emit a formatted table containing:

- **Frame index**: Current position in the sequence
- **KV entries**: Number of allocated cache entries
- **Cache hit-rate**: Percentage of cache reuses
- **Memory used**: MiB consumed by the cache

The print statements are wrapped in `if _KV_DEBUG:` guards to ensure they execute only when debugging is enabled. This design incurs minimal overhead—only string formatting and a `print` call—without altering the underlying KV-cache logic.

## When to Enable KV Debugging

Enable `LINGBOT_DEBUG_KV` when investigating specific runtime behaviors:

- **Unexpected GPU OOM errors**: Detect if the KV cache grows unbounded during long sequences
- **Sudden performance degradation**: Identify low hit-rates causing repeated recomputation
- **Cache reuse verification**: Confirm that new model variants correctly reuse KV entries across windows
- **Memory profiling**: Observe memory scaling across different input resolutions

## How to Enable KV Debugging

Set the environment variable before launching your Python script. The value determines the print interval.

### Basic Usage (Every Frame)

```bash
export LINGBOT_DEBUG_KV=1
python demo.py --input data/example/loop/

```

### Custom Interval (Every 10 Frames)

```bash
export LINGBOT_DEBUG_KV=10
python -m lingbot_map.demo --seq your_sequence.bin

```

### Capturing Output to File

```bash
export LINGBOT_DEBUG_KV=1
python demo.py 2>&1 | tee kv_debug.log

```

After capture, grep the log for `[KV DEBUG]` to extract statistics:

```bash
grep "\[KV DEBUG\]" kv_debug.log > stats_only.log

```

## Understanding the Debug Output

When enabled, the streaming transformer emits lines formatted as:

```

[KV DEBUG] frame=23  kv_entries=128  hit_rate=97.7%  mem=84.2MiB
[KV DEBUG] frame=24  kv_entries=128  hit_rate=98.0%  mem=84.2MiB

```

These metrics originate from the forward pass in [`gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream.py) and [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py). Monitor the `kv_entries` column for unbounded growth and the `hit_rate` column to verify cache efficiency. Stable `mem` values across frames indicate proper cache management, while monotonic increases suggest memory leaks.

## Key Source Files

The KV debugging implementation spans three critical files in the repository:

- **[`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py)**: Main streaming transformer containing the environment variable hook and primary printing logic
- **[`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py)**: Windowed streaming variant with interval parsing and conditional statistics output
- **[`lingbot_map/layers/flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/flashinfer_cache.py)**: Low-level KV-cache implementation that provides the raw statistics consumed by the debug output

## Summary

- **Set `LINGBOT_DEBUG_KV`** to a positive integer (e.g., `1`, `10`, `20`) to enable per-frame KV cache statistics in LingBot-Map
- **Leave it unset or empty** to disable debugging and eliminate runtime overhead
- **Monitor output** for `kv_entries`, `hit_rate`, and `mem` values to diagnose memory spikes and cache inefficiencies
- **Use file redirection** to capture logs for post-hoc analysis of long-running sequences
- **Minimal performance impact**: Debug printing only adds string formatting overhead without changing model behavior

## Frequently Asked Questions

### What values can I set for LINGBOT_DEBUG_KV?

Set `LINGBOT_DEBUG_KV` to any positive integer to specify the print interval. A value of `1` prints statistics every frame, while `10` prints every 10th frame. An empty string or unset variable disables output entirely. According to the source in [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py), the parser converts the string to an integer that controls the sampling frequency.

### Does enabling LINGBOT_DEBUG_KV slow down inference?

The debugging mechanism adds negligible overhead. The code only performs string formatting and `print` calls when the variable is set, and it does not modify the KV-cache operations or model weights. Performance impact is limited to the I/O cost of printing to stdout, which is typically insignificant compared to transformer computation.

### How do I distinguish between the streaming model variants when debugging?

[`gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream.py) implements the base streaming transformer, while [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py) handles windowed streaming with vocabulary-aware caching. Both respect `LINGBOT_DEBUG_KV`, but [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py) includes additional logic for parsing the interval. If you see window-specific statistics in the output, the model is likely using the windowed variant.

### Can I use LINGBOT_DEBUG_KV in production environments?

While technically possible, avoid setting `LINGBOT_DEBUG_KV` in production due to stdout pollution and potential log file bloat. The variable is designed for development and profiling scenarios. For production monitoring, instrument the [`flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/flashinfer_cache.py) layer directly with structured logging rather than relying on printed debug statements.