# Windowed Inference in LingBot-Map: Complete Parameter Guide for Long Video Processing

> Explore LingBot-Map's windowed inference parameters like window_size, overlap_keyframes, and flow_threshold. Optimize long video processing with efficient keyframe caching and alignment.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: api-reference
- Published: 2026-07-06

---

**The `inference_windowed` method in `GCTStream` accepts ten parameters—including `window_size`, `overlap_keyframes`, and `flow_threshold`—that control memory-bounded processing of long video sequences by managing keyframe caching and cross-window alignment.**

`LingBot-Map` (available at `Robbyant/lingbot-map`) implements a **windowed inference** mode that enables processing arbitrarily long video sequences while keeping GPU memory usage constant. This functionality centers on the `inference_windowed` method defined in **[`lingbot_map/models/gct_stream_window.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window.py)** (lines 590–714), which slices input streams into overlapping windows and manages keyframe retention through a sophisticated KV-cache eviction strategy.

## Core Parameters for Windowed Inference

The `inference_windowed` method signature accepts specific arguments that control how video frames are batched, cached, and stitched together across temporal boundaries.

### Input Configuration

- **`images`** (`torch.Tensor`): The input image batch with shape `[S, 3, H, W]` for single-batch inference or `[B, S, 3, H, W]` for batched processing. Pixel values must be normalized to the range `[0, 1]`.

### Window Structure

- **`window_size`** (`int`, default: `16`): Specifies the number of **keyframes** (frames whose KV-cache is retained) per window, including the scale frames. Increasing this value expands the temporal context available to the model but directly increases memory consumption.

- **`num_scale_frames`** (`int`, default: `None`): Determines how many initial frames in each window are processed as **scale frames** using bidirectional attention. These provide a reliable depth-scale reference for the remainder of the window. When `None`, the value falls back to `self.num_frame_for_scale`.

- **`scale_mode`** (`str`, default: `'median'`): Defines the strategy for estimating the inter-window scale factor. Currently, only `'median'` is implemented, which calculates the median depth-ratio across paired keyframes to align adjacent windows.

### Cross-Window Overlap

- **`overlap_size`** (`int`, default: `None`): Sets the length of overlapping *actual* frames between consecutive windows. If unspecified, it defaults to the number of scale frames (`num_scale_frames`). This overlap ensures the next window receives its scale-phase frames, maintaining prediction quality at boundaries.

- **`overlap_keyframes`** (`int`, default: `None`): Specifies the overlap expressed in **keyframes** rather than raw frames. This parameter takes precedence over `overlap_size` when `keyframe_interval > 1`. Internally, the method converts this to actual frames as `max(num_scale_frames, overlap_keyframes × keyframe_interval)`.

### Keyframe Management Strategy

- **`keyframe_interval`** (`int`, default: `1`): Controls the density of cached keyframes. A value of `1` means every frame becomes a keyframe (the original streaming behavior), while larger values reduce cache growth proportionally by retaining only every *N*-th frame.

- **`flow_threshold`** (`float`, default: `0.0`): Enables **flow-based dynamic keyframe detection** when set greater than zero. The method computes the mean optical-flow magnitude (in pixels) between the current frame and the last keyframe; frames exceeding this threshold become keyframes. When active, this mode overrides the fixed `keyframe_interval`.

- **`max_non_keyframe_gap`** (`int`, default: `30`): In flow-based mode, sets the maximum number of consecutive non-keyframes allowed before a keyframe is *forced* to prevent temporal drift.

### Memory Management

- **`output_device`** (`torch.device`, default: `None`): Specifies where final predictions are stored. When set to `torch.device('cpu')`, the method offloads per-window results from GPU memory, enabling processing of very long sequences on limited hardware. If `None`, predictions remain on the model's current device.

## Internal Processing Pipeline

Understanding how these parameters interact helps optimize inference for specific hardware constraints.

The method executes four distinct phases:

1. **Window Creation**: The algorithm first calculates the effective overlap (`eff_overlap`) based on `overlap_keyframes` (preferred) or `overlap_size`, then partitions the sequence into overlapping chunks.

2. **Scale Phase**: For each window, the first `num_scale_frames` frames are processed together with bidirectional attention. Their KV cache is preserved to establish the depth scale reference.

3. **Streaming Phase**: Remaining frames process sequentially. With `keyframe_interval > 1`, only every *N*-th frame enters the cache. When `flow_threshold > 0`, the method temporarily disables cache eviction via `_set_defer_eviction`, computes flow magnitude via `_compute_flow_magnitude`, and conditionally rolls back non-keyframes using `_rollback_last_frame`.

4. **Cross-Window Alignment**: After each window completes, `_align_and_stitch_windows` aligns predictions using the overlapping region via `_depth_ratio_scale` and pose-based similarity transforms, then stitches them via `_stitch_windows`.

## Implementation Examples

### Basic Windowed Inference

```python
import torch
from lingbot_map.models.gct_stream_window import GCTStream

# Initialize model (paths omitted for brevity)

model = GCTStream(
    img_size=518,
    patch_size=14,
    embed_dim=1024,
    pretrained_path="path/to/dinov2_weights.pt",
    enable_camera=True,
    enable_depth=True,
    enable_3d_rope=False,
)
model.eval()
model.cuda()

# Prepare dummy video: 120 frames of 518×518

B, S, C, H, W = 1, 120, 3, 518, 518
dummy_frames = torch.rand(B, S, C, H, W).cuda()  # Values in [0, 1]

# Run windowed inference

output = model.inference_windowed(
    images=dummy_frames,
    window_size=20,                # 20 keyframes per window

    overlap_keyframes=5,           # 5-keyframe overlap

    num_scale_frames=2,            # First 2 frames use bidirectional attention

    keyframe_interval=4,           # Cache every 4th frame

    flow_threshold=0.0,           # Disable flow-based mode

    output_device=torch.device('cpu'),  # Offload to CPU

)

# Extract results

pose_enc = output["pose_enc"]          # [B, S, 9]

depth = output.get("depth")            # [B, S, H, W, 1]

world_pts = output.get("world_points") # [B, S, H, W, 3]

```

### Flow-Based Dynamic Keyframe Detection

```python
output = model.inference_windowed(
    images=dummy_frames,
    window_size=16,
    overlap_size=4,              # 4-frame overlap

    flow_threshold=1.5,          # Activate flow-based mode (pixels)

    max_non_keyframe_gap=10,     # Force keyframe every 10 frames max

)

```

In this configuration, the model automatically selects keyframes based on optical flow magnitude while respecting the `max_non_keyframe_gap` constraint to prevent drift.

## Summary

- **`inference_windowed`** in [`lingbot_map/models/gct_stream_window.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window.py) enables fixed-memory processing of long videos through sliding windows.
- **Window boundaries** are controlled by `window_size`, `overlap_size`, and `overlap_keyframes`, with the latter preferred when subsampling keyframes.
- **Scale frames** (`num_scale_frames`) establish depth references using bidirectional attention within each window.
- **Keyframe density** is managed either statically via `keyframe_interval` or dynamically via `flow_threshold` and `max_non_keyframe_gap`.
- **Memory optimization** is achieved through `output_device` offloading, which moves results to CPU while keeping model weights on GPU.

## Frequently Asked Questions

### What is the difference between `overlap_size` and `overlap_keyframes`?

Both parameters define the overlap between consecutive windows, but `overlap_keyframes` operates in the keyframe domain while `overlap_size` operates in raw frames. When `keyframe_interval > 1`, use `overlap_keyframes` to ensure the overlap length respects the subsampled keyframe spacing. The method converts `overlap_keyframes` to actual frames using `max(num_scale_frames, overlap_keyframes × keyframe_interval)`.

### How does `flow_threshold` enable dynamic keyframe detection?

When `flow_threshold` is greater than zero, the model computes the mean optical-flow magnitude between the current frame and the lastkeyframe using `_compute_flow_magnitude`. If the displacement exceeds the threshold (in pixels), the frame becomes a keyframe regardless of `keyframe_interval`. This ensures that rapid motion or scene changes trigger cache retention, while static sequences maintain minimal memory usage.

### What happens if `num_scale_frames` exceeds `window_size`?

The `num_scale_frames` parameter must be less than or equal to `window_size` minus the required overlap. If configured improperly, the method may raise an error or fail to create valid windows because scale frames consume positions within the window budget. The overlap calculation ensures at least `num_scale_frames` are available for the scale phase while maintaining the specified overlap between windows.

### When should I use CPU offloading via `output_device`?

Set `output_device=torch.device('cpu')` when processing very long sequences (thousands of frames) that would otherwise exhaust GPU memory storing intermediate pose, depth, and point-cloud tensors. This offloads only the final prediction tensors to system RAM, allowing the model to remain on GPU for accelerated inference while preventing out-of-memory errors during aggregation.