# How to Debug KV Cache Issues in Ling Bot-Map Using the LINGBOT_DEBUG_KV Environment Variable

> Debug KV cache issues in Ling Bot-Map by setting the LINGBOT_DEBUG_KV environment variable to enable per-frame logging of KV cache statistics and memory usage.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-04

---

**TLDR:** Set the `LINGBOT_DEBUG_KV` environment variable to an integer value to enable per-frame logging of KV cache statistics—including entries, hit rates, and memory usage—from the streaming transformer models in Ling Bot-Map.

Ling Bot-Map manages attention key/value (KV) caching inside its transformer-based streaming models to optimize inference over long sequences. When diagnosing unexpected GPU OOM errors or sudden performance degradations, the **`LINGBOT_DEBUG_KV`** environment variable provides real-time visibility into cache behavior without modifying any model logic.

## How LINGBOT_DEBUG_KV Works Internally

The debug mechanism is implemented across the streaming model files, where the environment variable is read at import time and used to gate conditional logging inside the forward pass.

### Reading the Variable at Import

In [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py), the code reads the variable once when the module loads:

```python

# lingbot_map/models/gct_stream.py (line 24-25)

_KV_DEBUG = os.environ.get("LINGBOT_DEBUG_KV", "")

```

A similar pattern exists in [`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py), ensuring both the standard and windowed streaming variants support the same debugging interface.

### Parsing the Interval

The raw string value is parsed into an integer interval *N* that controls logging frequency. If the variable is empty or unset, debugging is disabled. When set to `1`, statistics print every frame; when set to `10`, output appears every 10th frame.

### Emitting Statistics During Forward Pass

Inside the model's forward pass, a conditional block checks `_KV_DEBUG` and prints a formatted statistics line. According to the source comments in [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py), the output includes:

- Frame index
- Number of KV entries allocated
- Cache hit-rate percentage
- Memory used (MiB)

The actual low-level cache data originates from [`lingbot_map/layers/flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/flashinfer_cache.py), which manages the underlying KV cache buffers.

## When to Enable KV Debugging

Enable this debugging mode when you encounter specific runtime issues:

- **Unexpected GPU OOM during long sequences** – Detect if the KV cache grows unbounded rather than reusing entries.
- **Sudden slowdown after specific frames** – Identify low hit-rates that force repeated recomputation of attention weights.
- **Verifying windowed cache behavior** – Confirm that [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py) correctly reuses KV entries across sliding windows.
- **Profiling memory scaling** – Observe how `mem` values correlate with input resolution changes.

## How to Enable and Use LINGBOT_DEBUG_KV

Export the variable before invoking any Python script that builds the model:

```bash

# Enable per-frame statistics (interval = 1)

export LINGBOT_DEBUG_KV=1
python demo.py --input data/example/loop/

```

```bash

# Reduce log verbosity by printing every 20 frames

export LINGBOT_DEBUG_KV=20
python -m lingbot_map.demo --seq your_sequence.bin

```

### Capturing Output to a Log File

Redirect both stdout and stderr to preserve the debug lines for analysis:

```bash
export LINGBOT_DEBUG_KV=1
python demo.py 2>&1 | tee kv_debug.log

```

You can then extract statistics using `grep`:

```bash
grep "\[KV DEBUG\]" kv_debug.log > kv_stats_only.log

```

## Understanding the Output Format

When enabled, the streaming transformers emit lines formatted as:

```

[KV DEBUG] frame=23  kv_entries=128  hit_rate=97.7%  mem=84.2MiB
[KV DEBUG] frame=24  kv_entries=128  hit_rate=98.0%  mem=84.2MiB

```

These values reflect the actual state of the cache managed by [`flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/flashinfer_cache.py). Stable `kv_entries` with high `hit_rate` values indicate healthy cache reuse, while monotonically increasing entries suggest memory leaks or improper cache clearing.

## Performance Considerations

The debug printing incurs minimal overhead—only string formatting and a `print` call. It does **not** modify KV-cache allocation strategy, window sizes, or model behavior, making it safe for use in production debugging sessions where you need to observe real memory pressure.

## Summary

- Set `LINGBOT_DEBUG_KV` to an integer *N* to print KV cache statistics every *N* frames.
- Statistics include frame index, entry count, hit rate percentage, and memory usage in MiB.
- Implemented in [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py) and [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py) with data sourced from [`lingbot_map/layers/flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/flashinfer_cache.py).
- Zero impact on model correctness; adds only observational overhead through conditional print statements.

## Frequently Asked Questions

### What values can I set for LINGBOT_DEBUG_KV?

Any positive integer. The value represents the interval between printed frames. Setting it to `1` outputs every frame, while `10` outputs every 10th frame. An empty or unset value disables the output entirely.

### Does enabling this variable change how the KV cache is allocated?

No. The variable is strictly observational. According to the source code in [`gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream.py), the debug logic wraps existing statistics in conditional print statements without modifying the underlying cache management in [`flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/flashinfer_cache.py) or altering the model's forward pass behavior.

### Why don't I see any output after setting the variable?

Ensure the variable is exported in your shell environment before the Python process starts. Since the code reads the variable at import time (line 24 in [`gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream.py)), setting it after importing the module will have no effect. Also verify you are running a script that instantiates the streaming transformer classes, as the logging occurs inside their forward methods.

### Can I use this to debug memory leaks in long-running sequences?

Yes. By logging the `mem` and `kv_entries` fields over time, you can correlate frame index with memory growth. If `kv_entries` increases monotonically without bound while processing a fixed window size, it indicates the cache is not releasing old entries, pointing to a management issue in the cache layer rather than a gradual system leak.