# When to Use GCTStreamWindow vs GCTStream for Your Dataset

> Learn when to use GCTStream vs GCTStreamWindow for your dataset. Optimize for accuracy with short sequences or memory efficiency with long videos.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: comparison
- Published: 2026-07-04

---

**Choose `GCTStream` for short-to-moderate sequences (≤200 frames) where full temporal context improves accuracy, and switch to `GCTStreamWindow` when processing long videos or deploying in memory-constrained environments that require constant `O(window)` memory usage.**

The `lingbot-map` repository by Robbyant provides streaming variants of the Global Context Transformer (GCT) designed for video pose estimation and depth prediction. Both `GCTStream` and `GCTStreamWindow` process frames incrementally using a key-value (KV) cache to avoid recomputing attention over the entire sequence, but they differ fundamentally in how they manage temporal context and memory consumption.

## Understanding the GCT Streaming Architecture

Both classes extend the base GCT architecture to support incremental inference. According to the source code in [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py), they leverage `AggregatorStream` to maintain a KV cache that persists across forward passes. This allows the model to process video streams frame-by-frame without storing the entire sequence in GPU memory simultaneously.

However, the critical distinction lies in the **KV cache eviction policy**. While `GCTStream` (defined in [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py)) retains all historical key-value pairs, `GCTStreamWindow` (in [`lingbot_map/models/gct_stream_window.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window.py)) implements a sliding-window mechanism that discards older frames to maintain constant memory usage.

## KV Cache Management: The Critical Difference

### GCTStream: Full Causal Attention

In `GCTStream`, the KV cache grows linearly with sequence length (`O(S)`). The class constructs its cache using `AggregatorStream` with `use_sdpa=False` to enable the FlashInfer backend, but **never evicts** old KV blocks. This provides full causal attention where every past frame remains attendable, maximizing temporal consistency for pose refinement and depth estimation.

### GCTStreamWindow: Sliding Window Attention

`GCTStreamWindow` overrides this behavior by passing `kv_cache_sliding_window` and related flags to the underlying `AggregatorStream`. As implemented in [`lingbot_map/models/gct_stream_window.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window.py), this bounds memory usage to `O(window)` regardless of video length. The cache retains only the most recent *N* KV blocks (default 64) plus optional **scale frames** configured via `kv_cache_scale_frames` and `kv_cache_cross_frame_special`.

## When to Use GCTStream for Maximum Temporal Context

Prefer `GCTStream` when your dataset contains **short-to-moderate sequences** (approximately ≤200 frames) and GPU memory permits linear growth. This approach is ideal when:

- You require the strongest possible temporal consistency across the entire video
- The scene complexity benefits from long-range attention (e.g., loop closure detection in visual odometry)
- You can afford the `O(S)` memory footprint during inference

Note that while `GCTStream` accepts `kv_cache_sliding_window` in its constructor, this parameter is **ignored** and the cache never evicts.

## When to Use GCTStreamWindow for Long Video Streaming

Switch to `GCTStreamWindow` when processing **very long videos** (thousands of frames) or deploying in **low-memory environments** such as single-GPU inference servers. This variant excels when:

- Video length would otherwise cause out-of-memory (OOM) errors with full caching
- Scene dynamics are locally consistent (recent context suffices)
- You need predictable, constant memory usage independent of sequence length

The bounded memory model allows processing hour-long recordings by tuning `kv_cache_sliding_window` (typically 64–128) to retain sufficient recent context while evicting older frames.

## Code Examples: Configuring Each Streaming Model

### Full Context Streaming with GCTStream

```python
from lingbot_map.models.gct_stream import GCTStream

# Optimal for videos ~150 frames

model = GCTStream(
    img_size=518,
    patch_size=14,
    embed_dim=1024,
    sliding_window_size=-1,          # Full causal attention

    enable_stream_inference=True,
    kv_cache_sliding_window=64,      # Ignored - cache never evicts

)

# Process frame-by-frame with KV cache accumulation

preds = model.inference_streaming(video_tensor, keyframe_interval=1)

```

### Sliding Window Streaming with GCTStreamWindow

```python
from lingbot_map.models.gct_stream_window import GCTStreamWindow

# Optimal for long videos ~3000+ frames

model = GCTStreamWindow(
    img_size=518,
    patch_size=14,
    embed_dim=1024,
    sliding_window_size=-1,          # Feed full sequence to model

    kv_cache_sliding_window=64,      # Retain only last 64 KV blocks

    kv_cache_scale_frames=8,         # Keep 8 scale frames for stability

    kv_cache_cross_frame_special=True,
    enable_stream_inference=True,
)

# Memory stays constant regardless of video length

preds = model.inference_streaming(video_tensor, keyframe_interval=5)

```

In the second example, GPU memory consumption remains roughly constant at `64 × (embedding_dim × patches)` bytes even for thousands of frames, whereas the first example scales linearly with frame count.

## Source Code Reference

The implementation differences between these streaming variants are located in the following files:

- **[`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py)** – Core streaming model with unbounded KV cache and full causal attention.
- **[`lingbot_map/models/gct_stream_window.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window.py)** – Adds sliding-window eviction via `kv_cache_sliding_window` configuration.
- **[`lingbot_map/models/gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window_v2.py)** – Refined implementation introducing additional cache management flags.
- **[`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py)** – Contains `AggregatorStream`, the base component managing KV cache storage and FlashInfer backend integration.
- **[`lingbot_map/heads/camera_head.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/heads/camera_head.py)** – Provides the causal camera head utilized by both streaming architectures.

## Summary

- **Use `GCTStream`** when processing ≤200 frames with available GPU memory, prioritizing full temporal context and maximum accuracy.
- **Use `GCTStreamWindow`** when processing long videos or deploying with memory constraints, accepting bounded context for constant `O(window)` memory usage.
- **Key parameter**: `kv_cache_sliding_window` controls the window size (default 64) in `GCTStreamWindow` but is ignored in `GCTStream`.
- **Implementation**: Both rely on `AggregatorStream` in [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py), differing only in cache eviction policy.

## Frequently Asked Questions

### Does GCTStreamWindow lose accuracy compared to GCTStream?

`GCTStreamWindow` may exhibit slightly reduced long-range temporal consistency since it cannot attend to frames outside the sliding window. However, for slowly changing scenes or when combined with `kv_cache_scale_frames` to retain key reference frames, the accuracy loss is minimal compared to the memory savings achieved.

### What is the default KV cache size in GCTStreamWindow?

According to the source code in [`lingbot_map/models/gct_stream_window.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window.py), the default `kv_cache_sliding_window` is **64** blocks. You can increase this to 128 or higher if GPU memory allows and your application requires longer temporal context, or reduce it further to minimize memory footprint.

### Can I switch between GCTStream and GCTStreamWindow without retraining?

Yes, both classes share the same underlying architecture and weights. They are inference-time variants that differ only in KV cache management. You can initialize either model from the same checkpoint and switch based on your deployment constraints without modifying the trained parameters.

### Why does GCTStream accept kv_cache_sliding_window if it doesn't use it?

The parameter exists in `GCTStream` for API compatibility with the base streaming interface, but as implemented in [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py), the value is ignored and the cache maintains all historical frames. Only `GCTStreamWindow` and its variants in [`gct_stream_window.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window.py) and [`gct_stream_window_v2.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_stream_window_v2.py) actually utilize this parameter for cache eviction.