Performance Trade-offs Between Different LingBot-Map Configurations: A Complete Guide

LingBot-Map configurations balance inference speed, GPU memory consumption, and reconstruction quality through mode selection, keyframe interval tuning, backend optimization, and compilation flags, with streaming mode delivering maximum FPS for short sequences and windowed mode enabling unlimited video length without memory errors.

Configuring inference for the Robbyant/lingbot-map repository requires navigating a complex trade-off space between real-time performance and pose estimation accuracy. The framework exposes specific hardware-tuning knobs in demo.py that control how the KV-cache, attention backends, and camera refinement steps interact, directly impacting whether your pipeline runs at 5 FPS or 30 FPS on consumer GPUs.

Streaming vs Windowed Inference Modes

The --mode argument in demo.py lines 61-63 selects between two fundamentally different memory management strategies that represent the primary branching point for LingBot-Map configurations.

Streaming Mode for Short-to-Medium Sequences

Streaming processes frames one-by-one while maintaining a persistent KV-cache in GPU memory. This configuration eliminates the overhead of cache resets and achieves the highest throughput for sequences under approximately 3,000 frames. The implementation in lingbot_map/models/gct_stream.py keeps hidden states resident, allowing the model to exploit long-range temporal dependencies without re-computation.

Windowed Mode for Arbitrarily Long Videos

Windowed mode, implemented in lingbot_map/models/gct_stream_window.py, splits long sequences into overlapping windows that each initialize their own KV-cache. While this adds a constant overhead from extra forward passes, it prevents out-of-memory (OOM) errors on videos of any length. The demo.py logic at lines 92-99 handles the window boundary management, where --window_size and --overlap_keyframes ensure smooth pose transitions between segments.

Memory Optimization Through Keyframe Management

LingBot-Map reduces memory pressure through selective caching controlled by --keyframe_interval and --kv_cache_sliding_window.

Keyframe Interval Tuning (--keyframe_interval)

The keyframe interval determines how many frames pass between cached states. In demo.py lines 74-80, the script auto-selects an interval when the frame count exceeds 320, reducing KV-cache memory linearly by a factor of N (where N is the interval value). However, larger intervals degrade pose accuracy in texture-poor regions because the model loses access to intermediate hidden states.

KV Cache and Scale Frame Configuration

The total memory budget follows this relationship: memory = (num_scale_frames + kv_cache_sliding_window × keyframe_interval) × per-frame KV size.

• --kv_cache_sliding_window: Default 64 keyframes; increasing this linearly raises memory usage
• --num_scale_frames: Default 8 bidirectional initialization frames; higher values improve global scale estimation but consume permanent KV slots

When the product exceeds available VRAM, the runtime either falls back to CPU off-loading, reduces the keyframe interval, or switches to windowed mode to reset the cache every window_size frames.

Backend Selection and Compilation Tuning

FlashInfer vs SDPA (--use_sdpa)

The attention backend significantly impacts throughput. FlashInfer provides paged KV-cache management that is 2–3× faster than standard PyTorch SDPA for long sequences, while reducing memory fragmentation. The code automatically detects FlashInfer availability (README lines 15-26) and falls back to SDPA when undefined. FlashInfer removes copy-on-write costs, delivering near-linear FPS scaling with sequence length.

Torch Compile (--compile)

Enabling --compile triggers torch.compile on hot model blocks including the aggregator and patch-embed layers, plus CUDA-graph warm-up. This adds 30–60 seconds of initialization time but yields approximately 5 FPS additional throughput on 518×378 inputs (README lines 31-33). Critical limitation: Compile only functions in streaming mode because the compiled graph assumes a fixed KV-cache topology that windowed mode violates.

Accuracy vs Speed Trade-offs

Camera Refinement Steps (--camera_num_iterations)

The camera head runs iterative refinement passes controlled by --camera_num_iterations. Reducing this from the default to 1 cuts camera head computation cost by approximately 4× with only slight quality degradation (README lines 17-24). This represents the most direct latency reduction available for real-time applications.

Data Type and CPU Offloading

In demo.py lines 48-50, the framework automatically selects bfloat16 on Ampere+ GPUs (otherwise float16), offering modest speed gains and larger dynamic range. The --offload_to_cpu flag (enabled by default) moves per-frame predictions to CPU during inference, reducing peak GPU memory by the size of the output tensors—essential for 8-GB cards—at the cost of small PCIe transfer overhead. Disable with --no-offload_to_cpu if CPU memory is constrained.

Practical Configuration Examples

Baseline Streaming for Short Clips (< 320 frames)

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/university \
    --mask_sky

This configuration requires no manual KV-cache tuning; the script auto-selects --keyframe_interval=1 for maximum accuracy.

Memory-Constrained Streaming (10k frames)

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --video_path long_walk.mp4 \
    --fps 10 \
    --mode streaming \
    --keyframe_interval 5

Reduces KV memory by 5×, suitable for long sequences that still fit in VRAM. Monitor memory allocation in the log output at lines 71-73.

Windowed Inference for Very Long Sequences

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --video_path long_walk.mp4 \
    --fps 10 \
    --mode windowed \
    --window_size 128 \
    --overlap_keyframes 8 \
    --keyframe_interval 4

Each window holds 128 keyframes; with interval 4, a window covers approximately (8 scale + 120 × 4) = 488 actual frames. The 8-frame overlap preserves cross-window consistency.

Optimized for 8-GB GPUs

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/courthouse \
    --camera_num_iterations 1 \
    --offload_to_cpu

Minimizes GPU peak memory and accelerates the camera head by reducing refinement iterations.

Maximum Throughput with FlashInfer

pip install flashinfer-python
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/oxford \
    --mask_sky

FlashInfer is detected automatically; no additional flags required. Verify installation by checking for the performance table in the README lines 15-26.

Summary

• Streaming mode provides the fastest inference for sequences under 3,000 frames by keeping the KV-cache resident, while windowed mode enables unlimited video length through cache resets at the cost of extra forward passes.
• Keyframe interval acts as a linear memory reduction lever—doubling the interval halves memory usage but potentially degrades pose accuracy in low-texture scenes.
• FlashInfer delivers 2–3× speedup over SDPA through paged KV-cache management, making it essential for high-throughput applications.
• Torch compile adds 30–60 seconds of warm-up but yields ~5 FPS gains, restricted exclusively to streaming mode due to graph topology constraints.
• Camera refinement steps offer the highest ROI for latency reduction, with 1 iteration providing 4× speedup versus default settings.
• Memory budget follows the formula: (num_scale_frames + kv_cache_sliding_window × keyframe_interval) × per-frame KV size.

Frequently Asked Questions

When should I use streaming mode versus windowed mode in LingBot-Map?

Use streaming mode for videos under approximately 3,000 frames where the KV-cache fits entirely in GPU memory, as it avoids the overhead of cache resets and supports torch compilation. Use windowed mode for arbitrarily long videos or when memory constraints force periodic cache clearing, accepting the constant overhead of extra forward passes between windows.

How does the keyframe interval affect memory usage and accuracy?

Increasing --keyframe_interval from 1 to N reduces KV-cache memory linearly by a factor of N, as fewer frames store their hidden states. However, accuracy may degrade during long-range tracking in texture-poor scenes because the model cannot access intermediate frame representations. The system auto-adjusts this interval at 320+ frames in streaming mode, but manual tuning is required for windowed configurations.

What is the performance difference between FlashInfer and SDPA backends?

FlashInfer provides 2–3× higher throughput than PyTorch SDPA on long sequences by implementing paged KV-cache management that reduces memory fragmentation and eliminates copy-on-write overhead. SDPA serves as a compatibility fallback when FlashInfer is unavailable, requiring no additional dependencies but delivering significantly lower FPS on extended videos.

Why does the --compile flag only work with streaming mode?

The --compile flag applies torch.compile to model blocks assuming a fixed KV-cache topology that is only maintained in streaming mode. Windowed mode dynamically resets and reallocates the cache between segments, violating the static graph assumptions required for CUDA graph capture and compilation optimization. Attempting to use compile with windowed mode will result in runtime errors or fallback to eager execution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}