# Performance Trade-offs Between Different LingBot-Map Configurations: A Complete Guide

> Explore LingBot-Map performance trade-offs. Optimize inference speed, GPU memory, and quality by tuning modes, keyframes, and backend settings for your specific needs.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: performance
- Published: 2026-07-05

---

**LingBot-Map configurations balance inference speed, GPU memory consumption, and reconstruction quality through mode selection, keyframe interval tuning, backend optimization, and compilation flags, with streaming mode delivering maximum FPS for short sequences and windowed mode enabling unlimited video length without memory errors.**

Configuring inference for the Robbyant/lingbot-map repository requires navigating a complex trade-off space between real-time performance and pose estimation accuracy. The framework exposes specific hardware-tuning knobs in [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) that control how the KV-cache, attention backends, and camera refinement steps interact, directly impacting whether your pipeline runs at 5 FPS or 30 FPS on consumer GPUs.

## Streaming vs Windowed Inference Modes

The `--mode` argument in [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) [lines 61-63](https://github.com/Robbyant/lingbot-map/blob/main/demo.py#L61-L63) selects between two fundamentally different memory management strategies that represent the primary branching point for **LingBot-Map configurations**.

### Streaming Mode for Short-to-Medium Sequences

**Streaming** processes frames one-by-one while maintaining a persistent KV-cache in GPU memory. This configuration eliminates the overhead of cache resets and achieves the highest throughput for sequences under approximately 3,000 frames. The implementation in [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py) keeps hidden states resident, allowing the model to exploit long-range temporal dependencies without re-computation.

### Windowed Mode for Arbitrarily Long Videos

**Windowed** mode, implemented in [`lingbot_map/models/gct_stream_window.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window.py), splits long sequences into overlapping windows that each initialize their own KV-cache. While this adds a constant overhead from extra forward passes, it prevents out-of-memory (OOM) errors on videos of any length. The [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) logic at [lines 92-99](https://github.com/Robbyant/lingbot-map/blob/main/demo.py#L92-L99) handles the window boundary management, where `--window_size` and `--overlap_keyframes` ensure smooth pose transitions between segments.

## Memory Optimization Through Keyframe Management

LingBot-Map reduces memory pressure through selective caching controlled by `--keyframe_interval` and `--kv_cache_sliding_window`.

### Keyframe Interval Tuning (--keyframe_interval)

The **keyframe interval** determines how many frames pass between cached states. In [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) [lines 74-80](https://github.com/Robbyant/lingbot-map/blob/main/demo.py#L74-L80), the script auto-selects an interval when the frame count exceeds 320, reducing KV-cache memory linearly by a factor of *N* (where *N* is the interval value). However, larger intervals degrade pose accuracy in texture-poor regions because the model loses access to intermediate hidden states.

### KV Cache and Scale Frame Configuration

The total **memory budget** follows this relationship: `memory = (num_scale_frames + kv_cache_sliding_window × keyframe_interval) × per-frame KV size`. 

- `--kv_cache_sliding_window`: Default 64 keyframes; increasing this linearly raises memory usage
- `--num_scale_frames`: Default 8 bidirectional initialization frames; higher values improve global scale estimation but consume permanent KV slots

When the product exceeds available VRAM, the runtime either falls back to CPU off-loading, reduces the keyframe interval, or switches to windowed mode to reset the cache every `window_size` frames.

## Backend Selection and Compilation Tuning

### FlashInfer vs SDPA (--use_sdpa)

The attention backend significantly impacts throughput. **FlashInfer** provides paged KV-cache management that is **2–3× faster** than standard PyTorch SDPA for long sequences, while reducing memory fragmentation. The code automatically detects FlashInfer availability (README [lines 15-26](https://github.com/Robbyant/lingbot-map/blob/main/README.md#L15-L26)) and falls back to SDPA when undefined. FlashInfer removes copy-on-write costs, delivering near-linear FPS scaling with sequence length.

### Torch Compile (--compile)

Enabling `--compile` triggers `torch.compile` on hot model blocks including the aggregator and patch-embed layers, plus CUDA-graph warm-up. This adds **30–60 seconds** of initialization time but yields approximately **5 FPS additional throughput** on 518×378 inputs (README [lines 31-33](https://github.com/Robbyant/lingbot-map/blob/main/README.md#L31-L33)). **Critical limitation:** Compile only functions in `streaming` mode because the compiled graph assumes a fixed KV-cache topology that windowed mode violates.

## Accuracy vs Speed Trade-offs

### Camera Refinement Steps (--camera_num_iterations)

The camera head runs iterative refinement passes controlled by `--camera_num_iterations`. Reducing this from the default to **1** cuts camera head computation cost by approximately **4×** with only slight quality degradation (README [lines 17-24](https://github.com/Robbyant/lingbot-map/blob/main/README.md#L17-L24)). This represents the most direct latency reduction available for real-time applications.

### Data Type and CPU Offloading

In [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) [lines 48-50](https://github.com/Robbyant/lingbot-map/blob/main/demo.py#L48-L50), the framework automatically selects **bfloat16** on Ampere+ GPUs (otherwise float16), offering modest speed gains and larger dynamic range. The `--offload_to_cpu` flag (enabled by default) moves per-frame predictions to CPU during inference, reducing peak GPU memory by the size of the output tensors—essential for 8-GB cards—at the cost of small PCIe transfer overhead. Disable with `--no-offload_to_cpu` if CPU memory is constrained.

## Practical Configuration Examples

### Baseline Streaming for Short Clips (< 320 frames)

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/university \
    --mask_sky

```

This configuration requires no manual KV-cache tuning; the script auto-selects `--keyframe_interval=1` for maximum accuracy.

### Memory-Constrained Streaming (10k frames)

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --video_path long_walk.mp4 \
    --fps 10 \
    --mode streaming \
    --keyframe_interval 5

```

Reduces KV memory by 5×, suitable for long sequences that still fit in VRAM. Monitor memory allocation in the log output at [lines 71-73](https://github.com/Robbyant/lingbot-map/blob/main/demo.py#L71-L73).

### Windowed Inference for Very Long Sequences

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --video_path long_walk.mp4 \
    --fps 10 \
    --mode windowed \
    --window_size 128 \
    --overlap_keyframes 8 \
    --keyframe_interval 4

```

Each window holds 128 keyframes; with interval 4, a window covers approximately (8 scale + 120 × 4) = 488 actual frames. The 8-frame overlap preserves cross-window consistency.

### Optimized for 8-GB GPUs

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/courthouse \
    --camera_num_iterations 1 \
    --offload_to_cpu

```

Minimizes GPU peak memory and accelerates the camera head by reducing refinement iterations.

### Maximum Throughput with FlashInfer

```bash
pip install flashinfer-python
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/oxford \
    --mask_sky

```

FlashInfer is detected automatically; no additional flags required. Verify installation by checking for the performance table in the README [lines 15-26](https://github.com/Robbyant/lingbot-map/blob/main/README.md#L15-L26).

## Summary

- **Streaming mode** provides the fastest inference for sequences under 3,000 frames by keeping the KV-cache resident, while **windowed mode** enables unlimited video length through cache resets at the cost of extra forward passes.
- **Keyframe interval** acts as a linear memory reduction lever—doubling the interval halves memory usage but potentially degrades pose accuracy in low-texture scenes.
- **FlashInfer** delivers 2–3× speedup over SDPA through paged KV-cache management, making it essential for high-throughput applications.
- **Torch compile** adds 30–60 seconds of warm-up but yields ~5 FPS gains, restricted exclusively to streaming mode due to graph topology constraints.
- **Camera refinement steps** offer the highest ROI for latency reduction, with 1 iteration providing 4× speedup versus default settings.
- Memory budget follows the formula: `(num_scale_frames + kv_cache_sliding_window × keyframe_interval) × per-frame KV size`.

## Frequently Asked Questions

### When should I use streaming mode versus windowed mode in LingBot-Map?

Use **streaming mode** for videos under approximately 3,000 frames where the KV-cache fits entirely in GPU memory, as it avoids the overhead of cache resets and supports torch compilation. Use **windowed mode** for arbitrarily long videos or when memory constraints force periodic cache clearing, accepting the constant overhead of extra forward passes between windows.

### How does the keyframe interval affect memory usage and accuracy?

Increasing `--keyframe_interval` from 1 to *N* reduces KV-cache memory linearly by a factor of *N*, as fewer frames store their hidden states. However, accuracy may degrade during long-range tracking in texture-poor scenes because the model cannot access intermediate frame representations. The system auto-adjusts this interval at 320+ frames in streaming mode, but manual tuning is required for windowed configurations.

### What is the performance difference between FlashInfer and SDPA backends?

**FlashInfer** provides 2–3× higher throughput than PyTorch SDPA on long sequences by implementing paged KV-cache management that reduces memory fragmentation and eliminates copy-on-write overhead. SDPA serves as a compatibility fallback when FlashInfer is unavailable, requiring no additional dependencies but delivering significantly lower FPS on extended videos.

### Why does the --compile flag only work with streaming mode?

The `--compile` flag applies `torch.compile` to model blocks assuming a **fixed KV-cache topology** that is only maintained in streaming mode. Windowed mode dynamically resets and reallocates the cache between segments, violating the static graph assumptions required for CUDA graph capture and compilation optimization. Attempting to use compile with windowed mode will result in runtime errors or fallback to eager execution.