# Understanding the Relationship Between window_size, overlap_keyframes, and Actual Frame Coverage in Lingbot-Map

> Explore how window_size, overlap_keyframes, and keyframe_interval impact actual frame coverage in Lingbot-Map. Understand seamless window stitching for better results.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: deep-dive
- Published: 2026-07-29

---

**The relationship between `window_size`, `overlap_keyframes`, and actual frame coverage in Lingbot-Map is governed by `keyframe_interval` multiplication, where `window_size` counts keyframes but actual coverage depends on scale frames plus interval-expanded differences, while `overlap_keyframes` converts to actual frames via the same interval to ensure seamless window stitching.**

In the [Lingbot-Map](https://github.com/Robbyant/lingbot-map) repository, windowed inference manages memory-efficient processing of long video sequences by sliding a context window across frames. The parameters `window_size` and `overlap_keyframes` appear to count frames directly, but they actually operate on **keyframes** while the **actual frame coverage** depends on the `keyframe_interval` and scale frame configuration. Understanding this translation is critical for tuning inference on long-form video without gaps or redundant processing.

## How window_size Translates to Actual Frames

The `window_size` parameter does not represent actual video frames—it counts **keyframes** that remain in the KV cache (including mandatory scale frames).

According to the docstring in [`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py) (lines 43-47), the total number of actual frames covered by one window follows this calculation:

```

actual_frames = scale_frames + (window_size - scale_frames) × keyframe_interval

```

Where:
- **scale_frames** (`ws` in the source) are the base frames always kept in cache
- **(window_size - scale_frames)** represents the additional keyframe slots
- **keyframe_interval** (`kf` in the code) expands each keyframe slot to its representative frame count

For example, with `window_size=16`, `scale_frames=4`, and `keyframe_interval=2`, the window actually covers `4 + (12 × 2) = 28` actual frames, not 16.

## Resolving Overlap Parameters

Lingbot-Map accepts overlap specification through two mutually exclusive parameters resolved in a priority order (lines 93-106 of [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py)):

1. **`overlap_keyframes`** – Overlap expressed in **keyframes** (preferred modern interface)
2. **`overlap_size`** – Legacy parameter specifying overlap in **actual frames** (used only when `overlap_keyframes` is absent)

When `overlap_keyframes` is provided, the code converts it to actual frames at lines 56-60:

```python

# From lines 56-60: converting keyframe overlap to actual frame overlap

overlap_actual = overlap_keyframes × keyframe_interval

```

This conversion ensures that the overlap region contains at least `overlap_keyframes × keyframe_interval` actual frames, maintaining temporal continuity regardless of keyframe sampling density.

## Effective Overlap Calculation and Constraints

The final overlap value—stored as `eff_overlap`—undergoes resolution logic between lines 93-106:

**Priority resolution**:
- If `overlap_keyframes` exists: `eff_overlap = overlap_keyframes × keyframe_interval`
- Else if `overlap_size` exists: `eff_overlap = overlap_size`
- Else default: `eff_overlap = num_scale_frames`

**Safety constraints**:
- **Minimum bound**: The effective overlap is forced to at least `num_scale_frames` (line 98), ensuring the next window retains sufficient scale frames for reliable alignment regardless of user input.
- **Maximum bound**: The value is clamped to `S-1` (total sequence length minus one) at line 106 to prevent cursor overshoot.

The cursor advancement follows:

```python
cursor = max(cursor - eff_overlap, window_start + window_scale)

```

This means each new window starts `eff_overlap` frames back from the previous window's end, creating the stitching region.

## Complete Coverage Formula

The actual coverage per inference step combines window size and overlap:

```

Total unique coverage per window = scale_frames + (window_size - scale_frames) × keyframe_interval
Effective overlap in actual frames = max(overlap_keyframes × keyframe_interval, scale_frames)

```

When `overlap_keyframes` is set, the shared actual frames equal `overlap_keyframes × keyframe_interval` (subject to the minimum scale frame constraint). When using legacy `overlap_size`, the value applies directly without interval multiplication.

## Practical Implementation Examples

The following examples demonstrate how these parameters translate in [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) (lines 396-401) and the core inference logic:

```python

# Example 1: Default keyframe_interval=1

# 16 keyframes per window, 8 overlapping actual frames

pred = model.inference_windowed(
    images,               # Tensor shape [S, 3, H, W]

    window_size=16,
    overlap_keyframes=8,  # Maps directly to 8 actual frames when interval=1

)

# Example 2: Keyframe interval > 1 (e.g., interval=2)

# 16 keyframes per window with 4 keyframes overlap

# Actual overlap becomes 4 × 2 = 8 frames

pred = model.inference_windowed(
    images,
    window_size=16,
    overlap_keyframes=4,
    keyframe_interval=2,
)

# Example 3: Legacy overlap_size usage

# Explicit actual frame overlap without keyframe conversion

pred = model.inference_windowed(
    images,
    window_size=16,
    overlap_size=12,  # Exactly 12 actual frames overlap

)

```

In each case, the model internally resolves `eff_overlap` using the priority logic described above and stitches windows accordingly.

## Key Implementation Files

| File | Role |
|------|------|
| [`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py) | Core windowed inference implementation containing the resolution logic for `window_size`, `overlap_keyframes`, and `overlap_size` (lines 43-47, 53-60, 93-106). |
| [`lingbot_map/models/gct_stream_window.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window.py) | Previous implementation version providing historical context for the windowing logic. |
| [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) | Command-line interface exposing `--overlap_size` and `--overlap_keyframes` arguments (lines 396-401). |

## Summary

- **`window_size`** counts keyframes, not actual frames; actual coverage equals `scale_frames + (window_size - scale_frames) × keyframe_interval`.
- **`overlap_keyframes`** is multiplied by `keyframe_interval` to determine actual frame overlap, while `overlap_size` specifies actual frames directly.
- **Effective overlap** is determined by priority: `overlap_keyframes` > `overlap_size` > default `num_scale_frames`.
- **Safety constraints** enforce minimum overlap of `scale_frames` and maximum of `S-1` total frames.
- These relationships ensure that windowed inference in Lingbot-Map maintains temporal continuity while managing KV cache memory efficiently.

## Frequently Asked Questions

### What is the difference between overlap_keyframes and overlap_size?

**`overlap_keyframes`** measures overlap in keyframe units and is converted to actual frames by multiplying with `keyframe_interval`, while **`overlap_size`** measures overlap directly in actual frames without conversion. According to lines 53-60 of [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py), `overlap_keyframes` takes precedence when both are provided, and `overlap_size` serves as a legacy fallback.

### How does keyframe_interval affect actual frame coverage?

The **`keyframe_interval`** acts as a multiplier between keyframe counts and actual frame counts. As documented at lines 43-47, each keyframe beyond the scale frames represents `keyframe_interval` actual frames. When set to 1, keyframes and actual frames correspond 1:1; when greater than 1, a single keyframe stands for multiple actual frames, expanding both window coverage and overlap regions proportionally.

### Why is effective overlap clamped to S-1?

The effective overlap (`eff_overlap`) is clamped to `S-1` (total sequence length minus one) at line 106 to prevent the sliding cursor from attempting to start a new window beyond the sequence boundary or creating negative indexing. This ensures the window stepping logic remains valid regardless of parameter configuration or sequence length.

### What happens if overlap_keyframes is smaller than scale frames?

If the calculated overlap from `overlap_keyframes` would result in fewer actual frames than `num_scale_frames`, the code at line 98 forces `eff_overlap` to equal `num_scale_frames`. This guarantees that every new window retains the minimum required scale frames necessary for reliable spatial alignment, overriding smaller user-specified values to maintain inference quality.