Lingbot-Map Performance Considerations: Optimizing Speed, Memory, and Reconstruction Quality

Lingbot-map performance depends on balancing KV-cache memory, inference mode selection, and backend optimization, with streaming mode offering fastest speeds for short sequences and windowed mode enabling arbitrarily long videos without out-of-memory errors.

The Robbyant/lingbot-map repository implements a spatial reconstruction system with configurable inference pipelines. Understanding the core lingbot-map performance considerations allows you to optimize the trade-off between pose estimation accuracy, GPU memory consumption, and frames-per-second throughput based on your specific hardware constraints.

Inference Modes and Memory Architecture

Streaming Mode vs. Windowed Mode

Lingbot-map offers two primary inference configurations that fundamentally change how memory is managed during processing.

Streaming mode processes frames sequentially while maintaining a persistent KV cache. This approach delivers the highest throughput for short-to-medium sequences (under approximately 3,000 frames) because the cache remains resident in GPU memory. The mode selection logic is implemented in demo.py at lines 61–63.

Windowed mode divides long sequences into overlapping chunks, each with an independent KV cache. This configuration introduces constant overhead due to extra forward passes but prevents out-of-memory errors when processing arbitrarily long videos. The windowed implementation resets the cache between segments in lingbot_map/models/gct_stream_window.py.

Key Configuration Parameters

Keyframe Interval and KV-Cache Management

The --keyframe_interval parameter controls how many frames pass between cached keyframes after the initial scale frames. Non-keyframes are processed but their hidden states are discarded.

Larger intervals reduce KV-cache memory linearly by a factor of N, though this may degrade pose accuracy when the model requires long-range temporal context. In demo.py, the script automatically selects a keyframe interval when frame counts exceed 320 (lines 74–80). In windowed mode, this interval directly multiplies the effective length of each processing window (lines 92–99).

The --kv_cache_sliding_window parameter sets the maximum number of cached frames (keyframes plus scale frames). While larger windows improve context retention, they increase memory consumption linearly. The default of 64 keyframes provides a balanced starting point for most scenes.

Scale Frames and Camera Refinement

--num_scale_frames specifies the number of bidirectional initialization frames used for global scale estimation. More scale frames improve accuracy but consume additional KV slots. The default value of 8 represents the recommended trade-off.

--camera_num_iterations controls iterative refinement passes for the camera head. Reducing this to 1 decreases camera head computation by approximately 4× with minimal quality degradation, as documented in the README at lines 17–24.

Backend and Compilation Optimizations

FlashInfer vs. SDPA Attention Backends

The --use_sdpa flag selects between attention implementations. FlashInfer provides a paged KV-cache that delivers 2–3× faster performance on long sequences while reducing memory fragmentation. When FlashInfer is unavailable, the system automatically falls back to native PyTorch SDPA.

FlashInfer installation requires the flashinfer-python package and is detailed in the README at lines 15–26. The backend detection requires no additional flags; the system selects FlashInfer automatically when available.

Torch Compile and Mixed Precision

Enabling --compile activates torch.compile on hot model blocks and performs CUDA-graph warm-up. This adds approximately 30–60 seconds of initial warm-up time but yields roughly 5 FPS improvement on 518×378 inputs according to the README at lines 31–33.

Compilation only functions in streaming mode because the compiled graph assumes a fixed KV-cache topology. The runtime automatically selects bfloat16 precision on Ampere+ GPUs and float16 on older hardware, configured in demo.py at lines 48–50.

Memory Management Strategies

Calculating Memory Budget

The total memory requirement follows this relationship:

Memory budget = (num_scale_frames + kv_cache_sliding_window × keyframe_interval) × per-frame KV size

When this product exceeds available GPU memory, the runtime either falls back to CPU off-loading, reduces the keyframe interval, or switches to windowed mode (which resets the cache every window_size frames).

The --offload_to_cpu flag moves per-frame predictions to CPU during inference, reducing peak GPU memory usage significantly (essential for 8 GB cards) at the cost of small CPU-PCIe transfer overhead. This is enabled by default but can be disabled with --no-offload_to_cpu.

Practical Configuration Examples

Baseline Streaming for Short Sequences

For videos under 320 frames, use the default streaming configuration:

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/university \
    --mask_sky

The script automatically sets --keyframe_interval=1 for maximum accuracy.

Streaming with Keyframe Reduction

For long videos (e.g., 10,000 frames) in streaming mode:

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --video_path long_walk.mp4 \
    --fps 10 \
    --mode streaming \
    --keyframe_interval 5

This reduces KV memory by approximately 5×, with memory usage logged at lines 71–73.

Windowed Inference for Very Long Sequences

For sequences exceeding 10,000 frames:

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --video_path long_walk.mp4 \
    --fps 10 \
    --mode windowed \
    --window_size 128 \
    --overlap_keyframes 8 \
    --keyframe_interval 4

Each window holds 128 keyframes; with keyframe_interval=4, a window covers approximately (8 scale + 120 × 4) = 488 actual frames. The overlap_keyframes=8 parameter ensures smooth pose transitions between windows, as explained in the README at lines 94–101.

Optimized Inference on 8 GB GPUs

For memory-constrained hardware:

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/courthouse \
    --camera_num_iterations 1 \
    --offload_to_cpu

This configuration reduces peak GPU memory and accelerates the camera head.

Maximum Throughput with FlashInfer

For optimal performance on supported hardware:

pip install flashinfer-python
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/oxford \
    --mask_sky

FlashInfer is detected automatically without requiring explicit flags.

Summary

• Streaming mode provides highest FPS for sequences under 3,000 frames but requires sufficient GPU memory for the KV cache.
• Windowed mode enables processing arbitrarily long videos by splitting sequences into overlapping chunks with independent caches.
• Keyframe interval offers linear memory reduction but may impact reconstruction quality on texture-poor sequences.
• FlashInfer delivers 2–3× speedup over SDPA through paged KV-cache implementation.
• Torch compilation adds 30–60 seconds of warm-up but improves throughput by ~5 FPS in streaming mode only.
• Memory calculations must account for scale frames, keyframe intervals, and sliding window sizes to prevent out-of-memory errors.

Frequently Asked Questions

What is the maximum video length supported by lingbot-map?

In streaming mode, practical limits depend on your GPU memory and typically cap around 3,000 frames for standard configurations. Windowed mode supports arbitrarily long sequences by processing video in overlapping segments, making it suitable for 10,000+ frame videos. The windowed implementation in lingbot_map/models/gct_stream_window.py resets the KV cache between segments to avoid memory accumulation.

How do I reduce GPU memory usage without switching to windowed mode?

Increase the --keyframe_interval to cache fewer frames, reduce --kv_cache_sliding_window, or enable --offload_to_cpu to move predictions to system memory. For 8 GB GPUs, combining --camera_num_iterations 1 with --offload_to_cpu typically provides sufficient memory headroom while maintaining acceptable reconstruction quality.

Why does torch.compile only work in streaming mode?

The compiled CUDA graph assumes a fixed KV-cache topology that remains constant throughout inference. Streaming mode maintains this stable structure, whereas windowed mode dynamically resets caches between segments, invalidating the compiled graph assumptions. Attempting compilation in windowed mode will result in errors or silent fallback to eager execution.

Which attention backend provides the best performance for long sequences?

FlashInfer delivers superior performance for long sequences through its paged KV-cache implementation, offering 2–3× faster execution and reduced memory fragmentation compared to native PyTorch SDPA. FlashInfer is automatically selected when the flashinfer-python package is installed, falling back to SDPA only when unavailable. For benchmark comparisons between backends, use the gct_profile.py script included in the repository.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}