# How to Configure window_size and overlap_keyframes for Windowed Inference on Long Sequences in LingBot-Map

> Configure window_size and overlap_keyframes for windowed inference on long sequences in LingBot-Map. Maintain temporal continuity across windows for efficient processing.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-08

---

**Set `window_size` to define the number of keyframes processed per GPU-bound window and `overlap_keyframes` to ensure temporal continuity across windows, where the effective overlap equals `max(num_scale_frames, overlap_keyframes * keyframe_interval)`.**

LingBot-Map processes very long video sequences by splitting them into **windows** that are processed independently with a fresh KV-cache. This keeps GPU memory bounded while still allowing high-quality predictions across the whole sequence. Configuring `window_size` and `overlap_keyframes` correctly in the Robbyant/lingbot-map repository is essential for balancing memory constraints against temporal consistency in the reconstructed 3-D map.

## Understanding Windowed Inference in LingBot-Map

The core of this mechanism lives in [`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py) — specifically the `inference_windowed` method at line 1025. When processing sequences longer than what fits in GPU memory, the model divides the input into segments rather than processing the entire video at once.

### What window_size Controls

**`window_size` counts keyframes**, not raw frames. A window contains a set of *scale frames* processed with bidirectional attention (defaulting to `self.num_frame_for_scale`), plus the keyframes whose KV-cache is retained.

The actual number of frames processed in a window follows this formula:

```

actual_frames = scale_frames + (window_size - scale_frames) * keyframe_interval

```

where `keyframe_interval` controls how often a frame is stored as a keyframe (lines 1043-1048 in [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py)).

Increasing `window_size` gives the model more keyframes per window, improving quality at the cost of higher memory usage. Decreasing it reduces memory but may degrade temporal consistency between windows.

### The Role of overlap_keyframes

By default, overlap between consecutive windows is expressed in **actual frames** via the `overlap_size` argument (line 1053). Supplying `overlap_keyframes` switches the overlap to be expressed in **keyframes** instead (lines 1055-1058).

This parameter is particularly valuable when `keyframe_interval > 1`, because it guarantees that at least one keyframe appears in the overlapping region. This is required for reliable cross-window alignment and temporal consistency.

## How Overlap Calculation Works Internally

When `overlap_keyframes` is provided, the library converts the keyframe overlap to actual frames using:

```

effective_overlap = max(num_scale_frames, overlap_keyframes * keyframe_interval)

```

The code then clamps this value to at most the current window size (lines 1099-1102). This ensures the next window's scale phase always has enough frames to function correctly.

**`overlap_keyframes` takes precedence over `overlap_size`**; if both are supplied, the latter is ignored. The effective overlap is always at least the number of scale frames, ensuring the bidirectional attention mechanism works correctly across window boundaries.

## Configuration Guidelines for Different Scenarios

Use these settings as starting points based on your video length and GPU memory:

- **Very long sequences (≫ 1k frames)**: Use small `window_size` (e.g., 8-12) to keep per-window memory low; set `overlap_keyframes` approximately equal to `window_size` to guarantee overlap.
- **High-quality temporal consistency**: Select larger `window_size` (e.g., 16-24) and `overlap_keyframes` approximately `window_size // 2`.
- **Fast inference on limited GPU**: Reduce `window_size` and rely on the default `overlap_size` (which equals the number of scale frames).
- **Non-uniform motion (fast camera moves)**: Enable flow-based keyframe detection (`flow_threshold>0`) and keep `overlap_keyframes` modest (e.g., 2-4). The flow logic creates extra windows where needed.

## Code Examples

### Programmatic Configuration in Python

Configure windowed inference directly through the `inference_windowed` method:

```python
import torch
from lingbot_map.models.gct_stream_window_v2 import GCTStream

# Load a pretrained model

model = GCTStream(pretrained_path='path/to/checkpoint.pt')
model.eval()
model.to('cuda')

# Dummy long video: 2000 frames, 3×256×256 RGB

images = torch.rand(1, 2000, 3, 256, 256, device='cuda')

# Example 1: Small windows with keyframe overlap

preds = model.inference_windowed(
    images,
    window_size=12,               # 12 keyframes per window

    overlap_keyframes=6,          # 6 keyframes overlap → 12 actual frames (since keyframe_interval=1)

    keyframe_interval=1,          # Store every frame as a keyframe

    num_scale_frames=4,           # Use 4 scale frames per window

)

# Example 2: Larger keyframe interval with explicit overlap

preds = model.inference_windowed(
    images,
    window_size=16,
    keyframe_interval=4,          # Keep KV cache for every 4th frame

    overlap_keyframes=4,          # Overlap = 4 keyframes → 16 actual frames (4 × interval)

    num_scale_frames=4,
)

```

### Command-Line Interface Usage

The [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) script exposes these parameters through command-line arguments (lines 362-400):

```bash
python demo.py path/to/video.mp4 \
    --window-size 12 \
    --overlap-keyframes 6 \
    --keyframe-interval 1 \
    --num-scale-frames 4

```

These arguments map directly to the `inference_windowed` method parameters.

## Summary

- **`window_size` defines keyframe count**, not raw frame count, controlling GPU memory per window through the formula `actual_frames = scale_frames + (window_size - scale_frames) * keyframe_interval`.
- **`overlap_keyframes`** specifies overlap in keyframes rather than frames, converting to `effective_overlap = max(num_scale_frames, overlap_keyframes * keyframe_interval)`.
- **`overlap_keyframes` takes precedence** over `overlap_size` when both are provided.
- The effective overlap is always clamped to at least the number of scale frames to ensure proper bidirectional attention across windows.
- Adjust these parameters based on sequence length and GPU memory, using smaller windows with substantial overlap for very long sequences.

## Frequently Asked Questions

### What is the difference between window_size and overlap_keyframes?

**`window_size`** controls how many keyframes are processed in each independent window, directly affecting GPU memory usage. **`overlap_keyframes`** determines how many keyframes are shared between consecutive windows to maintain temporal consistency. While `window_size` defines the processing unit, `overlap_keyframes` ensures continuity across window boundaries by guaranteeing shared keyframes in the overlap region.

### How does keyframe_interval affect the actual window size?

The `keyframe_interval` parameter controls how frequently frames are stored as keyframes. The actual frame count per window follows `actual_frames = scale_frames + (window_size - scale_frames) * keyframe_interval`. When `keyframe_interval` is greater than 1, the actual number of frames processed exceeds the `window_size` value, as `window_size` counts only keyframes, not intermediate frames.

### Why should I use overlap_keyframes instead of overlap_size?

Use **`overlap_keyframes`** when `keyframe_interval > 1` to ensure that at least one keyframe appears in the overlapping region between windows. This is essential for reliable cross-window alignment because the KV-cache synchronization depends on shared keyframes. The default `overlap_size` (measured in actual frames) might not guarantee keyframe presence in the overlap when frames are subsampled.

### Where is the windowed inference logic implemented in the source code?

The windowed inference mechanism is implemented in **[`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py)** within the `inference_windowed` method starting at line 1025. The overlap calculation logic resides at lines 1099-1102, while parameter parsing for the CLI interface appears in **[`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py)** at lines 362-400.