# Understanding the `clean_kv_cache` Function in LingBot-Map: When to Manually Reset KV Caches

> Learn when to manually call LingBot-Map's clean_kv_cache function to reset KV caches, prevent stale attention, and avoid GPU memory issues in streaming models.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: internals
- Published: 2026-07-31

---

**The `clean_kv_cache` function resets all cached key-value (KV) tensors in streaming models, aggregators, and camera heads to prevent stale attention across video sequences and avoid GPU memory exhaustion.**

The LingBot-Map repository implements transformer-based video understanding with streaming inference, where the `clean_kv_cache` function plays a critical role in managing the lifecycle of attention caches. When processing sequential video frames, the model accumulates KV tensors in a paged cache to enable efficient causal attention, but this cache must be explicitly cleared between independent sequences to ensure correct predictions and prevent memory leaks.

## What Does the `clean_kv_cache` Function Do?

The `clean_kv_cache` function is a utility method implemented across several core classes in the LingBot-Map architecture. Its primary responsibility is to **reset all cached key-value tensors** that accumulate during streaming inference runs, effectively returning the model to a pristine state before processing new input.

### Role in the FlashInfer Architecture

Inside the model's transformer layers, the **FlashInfer KV cache manager** (`FlashInferKVCacheManager`) maintains a paged cache that stores the **K** and **V** tensors of previously processed frames. This enables **causal attention** across video frames without recomputing the entire sequence history. However, when transitioning to a new video sequence, the cache still holds tensors from the previous video.

If the cache is not cleared, the attention mechanism for the new video incorrectly attends to stale tokens from previous sequences, resulting in erroneous pose or depth predictions. Additionally, uncleared pages accumulate indefinitely, leading to GPU memory exhaustion.

### Components That Implement Cache Clearing

The `clean_kv_cache` method exists on multiple high-level components:

- **Streaming models**: `GCTStream`, `GCTStreamWindow`, and `GCTStreamWindowV2`
- **Aggregator**: `AggregatorStream`
- **Camera head**: `CameraCausalHead`

In [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py) (lines 84-90), the implementation traverses the model hierarchy and delegates to subcomponents:

```python
def clean_kv_cache(self):
    """Clean KV cache in aggregator."""
    if hasattr(self.aggregator, 'clean_kv_cache'):
        self.aggregator.clean_kv_cache()
    else:
        logger.warning("Aggregator does not support KV cache cleaning")
    if hasattr(self.camera_head, 'kv_cache'):
        self.camera_head.clean_kv_cache()
    else:
        logger.warning("Camera head does not support KV cache cleaning")

```

The underlying reset logic resides in [`lingbot_map/layers/flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/flashinfer_cache.py) (lines 23-33), where the `FlashInferKVCacheManager.reset()` method zeros all per-block deques and counters.

## When to Call `clean_kv_cache` Manually

While the high-level API handles cache clearing automatically in most cases, several scenarios require explicit manual invocation of the `clean_kv_cache` function.

### Switching Between Video Sequences

When processing a **new video or image sequence**, you must clear the KV cache to prevent cross-contamination between videos. If stale KV tensors persist, the model's attention mechanism references tokens from previous sequences, producing incorrect predictions.

**Recommended call site**: At the beginning of your inference loop before processing the first frame of a new video.

### Running Multiple Independent Inference Passes

If you are **running multiple independent inference passes within the same Python process**, manual cache clearing is essential between runs. This prevents information leakage between unrelated videos when you want a fresh cache for each inference pass.

**Recommended call site**: Immediately before each `model.forward()` or `model.inference_streaming()` call if you bypass the high-level streaming API.

### Memory Management and Debugging

After **long streaming sessions**, calling `clean_kv_cache` frees GPU memory that is no longer needed, preventing out-of-memory (OOM) errors during extended operation. During **debugging or profiling**, manual cache clearing allows you to isolate the effects of KV cache reuse and measure baseline performance without accumulated state.

**Note**: The `inference_streaming` method in `GCTStream` already invokes `self.clean_kv_cache()` automatically after moving input to the device (see line 106). You only need manual calls when using low-level APIs or when resetting the cache mid-stream (for example, after a deliberate scene change).

## Code Implementation and File Locations

The cache clearing mechanism spans four critical files in the repository:

| File | Role |
|------|------|
| [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py) | Defines `GCTStream.clean_kv_cache` and invokes it in `inference_streaming` |
| [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py) | Implements `AggregatorStream.clean_kv_cache`, calling the FlashInfer manager |
| [`lingbot_map/heads/camera_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/camera_head.py) | Provides `CameraCausalHead.clean_kv_cache` for per-camera caches |
| [`lingbot_map/layers/flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/flashinfer_cache.py) | Core FlashInfer KV cache manager; `reset()` empties internal deques |

The `FlashInferKVCacheManager.reset()` method (lines 23-33) performs the actual memory reset by clearing the underlying data structures used for paged attention.

## Practical Usage Examples

### Example 1: High-Level API Usage

When using `inference_streaming`, the cache clears automatically:

```python
model = GCTStream(...)
predictions = model.inference_streaming(video_tensor)   # clean_kv_cache() called internally

```

### Example 2: Low-Level Forward Calls

When using `forward()` directly, you must manually manage the cache lifecycle:

```python
model = GCTStream(...)
model.clean_kv_cache()          # explicit reset before first frame

for frame in video_frames:
    out = model.forward(frame)  # KV cache grows as frames process

# Optional reset after sequence

model.clean_kv_cache()

```

### Example 3: Handling Scene Changes

Reset the cache mid-stream when processing distinct segments:

```python
for segment in segments:
    model.clean_kv_cache()          # clear before new segment

    seg_pred = model.inference_streaming(segment)
    # process predictions...

```

## Summary

- The `clean_kv_cache` function resets **KV tensors** in `GCTStream`, `AggregatorStream`, and `CameraCausalHead` to prevent attention contamination.
- Call it manually when **switching videos**, **running multiple independent inferences**, or **debugging** to ensure state isolation.
- The high-level `inference_streaming` method automatically clears the cache at line 106, but low-level `forward()` usage requires explicit management.
- The underlying implementation resides in `FlashInferKVCacheManager.reset()` within [`lingbot_map/layers/flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/flashinfer_cache.py).

## Frequently Asked Questions

### What happens if I don't call `clean_kv_cache` between videos?

The model's attention mechanism will attend to **stale KV tensors** from previous sequences, causing incorrect pose and depth predictions. Additionally, GPU memory will accumulate uncleared pages, eventually causing out-of-memory errors.

### Does `inference_streaming` call `clean_kv_cache` automatically?

Yes. The `inference_streaming` method in [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py) invokes `self.clean_kv_cache()` automatically after moving input to the device (around line 106). You only need manual calls when bypassing this high-level API or resetting mid-stream.

### Which components have their own `clean_kv_cache` methods?

The method is implemented on **three architectural levels**: the streaming models (`GCTStream` and variants), the aggregator (`AggregatorStream`), and the camera head (`CameraCausalHead`). The high-level model method delegates to these subcomponents to ensure complete cache clearance.

### How does `clean_kv_cache` relate to FlashInfer?

The function serves as a **wrapper** around `FlashInferKVCacheManager.reset()`. While `clean_kv_cache` handles the high-level orchestration across model components, the actual memory reset occurs in [`lingbot_map/layers/flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/flashinfer_cache.py) where the paged attention buffers are zeroed and deques are cleared.