How to Optimize Performance in LingBot-Map: 8 Proven Strategies for Faster 3D Reconstruction

Enable the FlashInfer backend with --backend flashinfer, reduce the KV-cache window using --keyframe-interval and windowed mode, and compile the model with gct_profile.py to achieve ~20 FPS on long video sequences while keeping GPU memory under 2 GB.

LingBot-Map is a feed-forward 3D reconstruction model that relies on a Geometric Context Transformer to process video streams. To optimize performance in LingBot-Map, you must minimize memory bandwidth bottlenecks in the KV-cache and eliminate redundant computation in the transformer layers. The following strategies target the core performance hotspots identified in the lingbot_map/layers/flashinfer_cache.py and lingbot_map/layers/vision_transformer.py modules.

Use the FlashInfer Backend for Paged KV-Cache Management

The FlashInfer backend provides a paged KV-cache implementation that avoids copying the entire cache each step, cutting both latency and memory consumption. When FlashInfer is not installed, the model falls back to PyTorch's native SDPA, which is significantly slower on long sequences.

Install the backend and enable it explicitly:

pip install --index-url https://pypi.org/simple flashinfer-python
python demo.py --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/university \
    --backend flashinfer

The FlashInferKVCacheManager in lingbot_map/layers/flashinfer_cache.py builds a two-stream paged layout separating patch pages from special tokens. By recycling patch pages and never evicting the special token stream, the manager keeps the visible page table compact (approximately 88 pages for a 64-frame window) and reuses the same plan for all layers, eliminating per-layer kernel launches.

Reduce the KV-Cache Window with Keyframe Intervals

The cache size grows linearly with the sliding window (sliding_window parameter defaults to 64). For sequences longer than the training range (~320 frames), the KV-cache becomes a memory bottleneck.

Control the cache growth using these flags:

• --keyframe_interval N – Retain only every N-th frame in the KV-cache. Non-keyframes still produce predictions but do not consume cache slots.
• --mode windowed --window_size W – Reset the cache every W slots, enabling arbitrarily long runs.
• --overlap_keyframes 8 – Share context between windows to maintain temporal coherence.

Run a 10,000-frame video with 2-frame keyframes:

python demo.py \
    --image_folder path/to/long_video_frames \
    --model_path /path/to/lingbot-map-long.pt \
    --keyframe_interval 2 \
    --mode windowed --window_size 128 \
    --overlap_keyframes 8

The windowed mode triggers FlashInferKVCacheManager.evict_frames only after the sliding window exceeds the configured limit, while keyframe_interval caps the number of cached frames, keeping memory roughly constant regardless of sequence length.

Compile the Model for Target Hardware

LingBot-Map includes a JIT compilation helper in gct_profile.py that optimizes attention kernels for your specific CUDA architecture. Compiled models eliminate Python overhead and allow FlashInfer to run at peak speed.

Compile for your GPU:

python gct_profile.py --backend flashinfer --dtype bf16 --compile

The compiled model is automatically loaded by demo.py on subsequent runs. The profiling script also reports per-layer runtime, helping identify remaining hotspots in lingbot_map/layers/attention.py.

Use Lower-Precision Data Types

When FlashInfer is active, the cache stores tensors in bfloat16 (or fp16). You can force fp32 for debugging, but for best performance, use the default lower-precision mode:

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --dtype bf16

The dtype argument propagates to FlashInferKVCacheManager where self.dtype is set in the constructor, halving the memory bandwidth requirements compared to fp32.

Offload Per-Frame Predictions to CPU

When GPU memory is constrained, the --offload_to_cpu flag moves dense point-cloud tensors off-GPU after each frame. This is enabled by default, but you can explicitly toggle it:

# Enable offloading (default, saves VRAM)

python demo.py --image_folder example/university --offload_to_cpu

# Disable offloading (if you have spare GPU memory)

python demo.py --image_folder example/university --no-offload_to_cpu

Disabling offloading can shave a few milliseconds per frame by eliminating CPU-GPU transfer overhead, but requires approximately 500MB additional VRAM for typical 518×378 inputs.

Reduce Iterative Camera Refinement

The camera head in lingbot_map/layers/vision_transformer.py performs camera_num_iterations (default 4) refinement passes per frame. Each iteration triggers a full forward pass through the transformer, making this a linear cost multiplier.

Reduce iterations for maximum speed:

python demo.py \
    --image_folder example/university \
    --model_path /path/to/lingbot-map-long.pt \
    --camera_num_iterations 1

Setting --camera_num_iterations 1 reduces transformer executions by 75% at a modest cost to pose accuracy, ideal for real-time preview processing.

Pre-compute Sky Masks to Eliminate Segmentation Overhead

Sky masking adds a forward pass through an ONNX segmentation model. Cache these masks once and reuse them across runs:

python demo.py \
    --image_folder example/oxford \
    --mask_sky \
    --sky_mask_dir ./oxford_sky_masks

Masks are saved under <image_folder>_sky_masks/ and loaded instantly on subsequent runs, bypassing the segmentation model entirely.

Profile and Tune with gct_profile.py

Use the profiling script to measure FPS across different resolutions, window sizes, and backends:

python gct_profile.py \
    --backend flashinfer \
    --dtype bf16 \
    --keyframe_interval 2 \
    --window_size 128

The output includes a breakdown by module (attention, KV-cache, camera head), enabling data-driven decisions when you optimize performance in LingBot-Map for your specific hardware.

Summary

• Install FlashInfer (pip install flashinfer-python) and use --backend flashinfer to enable paged KV-cache management that eliminates per-layer kernel launches.
• Reduce cache pressure with --keyframe-interval 2 and --mode windowed --window_size 128 to keep memory under 2 GB for 10,000+ frame sequences.
• Compile the model using gct_profile.py --compile to optimize attention kernels for your CUDA architecture.
• Lower precision to bf16 to halve memory bandwidth usage.
• Minimize camera iterations to 1 for maximum throughput when pose accuracy is less critical than speed.
• Cache sky masks offline to avoid repeated ONNX segmentation inference.

Frequently Asked Questions

What is the fastest backend for LingBot-Map inference?

FlashInfer is the fastest backend. According to the source code in lingbot_map/layers/flashinfer_cache.py, FlashInfer implements a two-stream paged KV-cache that avoids copying the entire cache each step, while the fallback PyTorch SDPA requires expensive memory copies on long sequences. Use --backend flashinfer after installing flashinfer-python.

How do I run LingBot-Map on videos longer than 10,000 frames without running out of memory?

Use windowed mode with a keyframe interval. Combine --mode windowed --window_size 128 with --keyframe-interval 2 to reset the KV-cache periodically while retaining only every second frame in the cache. This keeps GPU memory roughly constant regardless of video length, as implemented in FlashInferKVCacheManager.evict_frames.

Why is my GPU memory usage so high during inference?

High memory usage typically stems from the KV-cache growing with sequence length or camera refinement iterations. Reduce --camera_num_iterations from 4 to 1 to eliminate three full transformer passes per frame. Additionally, ensure --offload_to_cpu is enabled (default) to move point-cloud tensors off the GPU immediately after generation.

Does lowering camera iterations affect reconstruction accuracy?

Yes, reducing --camera_num_iterations from the default 4 to 1 will modestly reduce pose accuracy because the camera head performs iterative refinement. However, the first iteration already provides a reasonable estimate, and the speed improvement (4× fewer transformer passes) is often worth the trade-off for real-time applications or preview processing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}