# Lingbot-Map Performance Considerations: Optimizing Speed, Memory, and Reconstruction Quality

> Optimize lingbot-map performance by balancing KV-cache memory, inference mode, and backend settings. Choose streaming for speed or windowed for long videos.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: performance
- Published: 2026-07-19

---

**Lingbot-map performance depends on balancing KV-cache memory, inference mode selection, and backend optimization, with streaming mode offering fastest speeds for short sequences and windowed mode enabling arbitrarily long videos without out-of-memory errors.**

The `Robbyant/lingbot-map` repository implements a spatial reconstruction system with configurable inference pipelines. Understanding the core lingbot-map performance considerations allows you to optimize the trade-off between pose estimation accuracy, GPU memory consumption, and frames-per-second throughput based on your specific hardware constraints.

## Inference Modes and Memory Architecture

### Streaming Mode vs. Windowed Mode

Lingbot-map offers two primary inference configurations that fundamentally change how memory is managed during processing.

**Streaming mode** processes frames sequentially while maintaining a persistent KV cache. This approach delivers the highest throughput for short-to-medium sequences (under approximately 3,000 frames) because the cache remains resident in GPU memory. The mode selection logic is implemented in [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) at lines [61–63](https://github.com/Robbyant/lingbot-map/blob/main/demo.py#L61-L63).

**Windowed mode** divides long sequences into overlapping chunks, each with an independent KV cache. This configuration introduces constant overhead due to extra forward passes but prevents out-of-memory errors when processing arbitrarily long videos. The windowed implementation resets the cache between segments in [`lingbot_map/models/gct_stream_window.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window.py).

## Key Configuration Parameters

### Keyframe Interval and KV-Cache Management

The **`--keyframe_interval`** parameter controls how many frames pass between cached keyframes after the initial scale frames. Non-keyframes are processed but their hidden states are discarded.

Larger intervals reduce KV-cache memory linearly by a factor of N, though this may degrade pose accuracy when the model requires long-range temporal context. In [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py), the script automatically selects a keyframe interval when frame counts exceed 320 (lines [74–80](https://github.com/Robbyant/lingbot-map/blob/main/demo.py#L74-L80)). In windowed mode, this interval directly multiplies the effective length of each processing window (lines [92–99](https://github.com/Robbyant/lingbot-map/blob/main/demo.py#L92-L99)).

The **`--kv_cache_sliding_window`** parameter sets the maximum number of cached frames (keyframes plus scale frames). While larger windows improve context retention, they increase memory consumption linearly. The default of 64 keyframes provides a balanced starting point for most scenes.

### Scale Frames and Camera Refinement

**`--num_scale_frames`** specifies the number of bidirectional initialization frames used for global scale estimation. More scale frames improve accuracy but consume additional KV slots. The default value of 8 represents the recommended trade-off.

**`--camera_num_iterations`** controls iterative refinement passes for the camera head. Reducing this to 1 decreases camera head computation by approximately 4× with minimal quality degradation, as documented in the README at lines [17–24](https://github.com/Robbyant/lingbot-map/blob/main/README.md#L17-L24).

## Backend and Compilation Optimizations

### FlashInfer vs. SDPA Attention Backends

The **`--use_sdpa`** flag selects between attention implementations. **FlashInfer** provides a paged KV-cache that delivers 2–3× faster performance on long sequences while reducing memory fragmentation. When FlashInfer is unavailable, the system automatically falls back to native PyTorch SDPA.

FlashInfer installation requires the `flashinfer-python` package and is detailed in the README at lines [15–26](https://github.com/Robbyant/lingbot-map/blob/main/README.md#L15-L26). The backend detection requires no additional flags; the system selects FlashInfer automatically when available.

### Torch Compile and Mixed Precision

Enabling **`--compile`** activates `torch.compile` on hot model blocks and performs CUDA-graph warm-up. This adds approximately 30–60 seconds of initial warm-up time but yields roughly 5 FPS improvement on 518×378 inputs according to the README at lines [31–33](https://github.com/Robbyant/lingbot-map/blob/main/README.md#L31-L33).

Compilation only functions in streaming mode because the compiled graph assumes a fixed KV-cache topology. The runtime automatically selects **bfloat16** precision on Ampere+ GPUs and **float16** on older hardware, configured in [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) at lines [48–50](https://github.com/Robbyant/lingbot-map/blob/main/demo.py#L48-L50).

## Memory Management Strategies

### Calculating Memory Budget

The total memory requirement follows this relationship:

**Memory budget = (`num_scale_frames` + `kv_cache_sliding_window` × `keyframe_interval`) × per-frame KV size**

When this product exceeds available GPU memory, the runtime either falls back to CPU off-loading, reduces the keyframe interval, or switches to windowed mode (which resets the cache every `window_size` frames).

The **`--offload_to_cpu`** flag moves per-frame predictions to CPU during inference, reducing peak GPU memory usage significantly (essential for 8 GB cards) at the cost of small CPU-PCIe transfer overhead. This is enabled by default but can be disabled with `--no-offload_to_cpu`.

## Practical Configuration Examples

### Baseline Streaming for Short Sequences

For videos under 320 frames, use the default streaming configuration:

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/university \
    --mask_sky

```

The script automatically sets `--keyframe_interval=1` for maximum accuracy.

### Streaming with Keyframe Reduction

For long videos (e.g., 10,000 frames) in streaming mode:

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --video_path long_walk.mp4 \
    --fps 10 \
    --mode streaming \
    --keyframe_interval 5

```

This reduces KV memory by approximately 5×, with memory usage logged at lines [71–73](https://github.com/Robbyant/lingbot-map/blob/main/demo.py#L71-L73).

### Windowed Inference for Very Long Sequences

For sequences exceeding 10,000 frames:

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --video_path long_walk.mp4 \
    --fps 10 \
    --mode windowed \
    --window_size 128 \
    --overlap_keyframes 8 \
    --keyframe_interval 4

```

Each window holds 128 keyframes; with `keyframe_interval=4`, a window covers approximately (8 scale + 120 × 4) = 488 actual frames. The `overlap_keyframes=8` parameter ensures smooth pose transitions between windows, as explained in the README at lines [94–101](https://github.com/Robbyant/lingbot-map/blob/main/README.md#L94-L101).

### Optimized Inference on 8 GB GPUs

For memory-constrained hardware:

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/courthouse \
    --camera_num_iterations 1 \
    --offload_to_cpu

```

This configuration reduces peak GPU memory and accelerates the camera head.

### Maximum Throughput with FlashInfer

For optimal performance on supported hardware:

```bash
pip install flashinfer-python
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/oxford \
    --mask_sky

```

FlashInfer is detected automatically without requiring explicit flags.

## Summary

- **Streaming mode** provides highest FPS for sequences under 3,000 frames but requires sufficient GPU memory for the KV cache.
- **Windowed mode** enables processing arbitrarily long videos by splitting sequences into overlapping chunks with independent caches.
- **Keyframe interval** offers linear memory reduction but may impact reconstruction quality on texture-poor sequences.
- **FlashInfer** delivers 2–3× speedup over SDPA through paged KV-cache implementation.
- **Torch compilation** adds 30–60 seconds of warm-up but improves throughput by ~5 FPS in streaming mode only.
- **Memory calculations** must account for scale frames, keyframe intervals, and sliding window sizes to prevent out-of-memory errors.

## Frequently Asked Questions

### What is the maximum video length supported by lingbot-map?

In **streaming mode**, practical limits depend on your GPU memory and typically cap around 3,000 frames for standard configurations. **Windowed mode** supports arbitrarily long sequences by processing video in overlapping segments, making it suitable for 10,000+ frame videos. The windowed implementation in [`lingbot_map/models/gct_stream_window.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream_window.py) resets the KV cache between segments to avoid memory accumulation.

### How do I reduce GPU memory usage without switching to windowed mode?

Increase the **`--keyframe_interval`** to cache fewer frames, reduce **`--kv_cache_sliding_window`**, or enable **`--offload_to_cpu`** to move predictions to system memory. For 8 GB GPUs, combining `--camera_num_iterations 1` with `--offload_to_cpu` typically provides sufficient memory headroom while maintaining acceptable reconstruction quality.

### Why does torch.compile only work in streaming mode?

The compiled CUDA graph assumes a **fixed KV-cache topology** that remains constant throughout inference. Streaming mode maintains this stable structure, whereas windowed mode dynamically resets caches between segments, invalidating the compiled graph assumptions. Attempting compilation in windowed mode will result in errors or silent fallback to eager execution.

### Which attention backend provides the best performance for long sequences?

**FlashInfer** delivers superior performance for long sequences through its paged KV-cache implementation, offering 2–3× faster execution and reduced memory fragmentation compared to native PyTorch SDPA. FlashInfer is automatically selected when the `flashinfer-python` package is installed, falling back to SDPA only when unavailable. For benchmark comparisons between backends, use the [`gct_profile.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_profile.py) script included in the repository.