# How to Optimize Performance in LingBot-Map: 8 Proven Strategies for Faster 3D Reconstruction

> Boost LingBot-Map performance with 8 proven strategies. Achieve faster 3D reconstruction and reduce GPU memory to under 2 GB with this guide. Optimize your results now.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: performance
- Published: 2026-07-06

---

**Enable the FlashInfer backend with `--backend flashinfer`, reduce the KV-cache window using `--keyframe-interval` and windowed mode, and compile the model with [`gct_profile.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_profile.py) to achieve ~20 FPS on long video sequences while keeping GPU memory under 2 GB.**

LingBot-Map is a feed-forward 3D reconstruction model that relies on a Geometric Context Transformer to process video streams. To optimize performance in LingBot-Map, you must minimize memory bandwidth bottlenecks in the KV-cache and eliminate redundant computation in the transformer layers. The following strategies target the core performance hotspots identified in the [`lingbot_map/layers/flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/flashinfer_cache.py) and [`lingbot_map/layers/vision_transformer.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/vision_transformer.py) modules.

## Use the FlashInfer Backend for Paged KV-Cache Management

The **FlashInfer** backend provides a paged KV-cache implementation that avoids copying the entire cache each step, cutting both latency and memory consumption. When FlashInfer is not installed, the model falls back to PyTorch's native SDPA, which is significantly slower on long sequences.

Install the backend and enable it explicitly:

```bash
pip install --index-url https://pypi.org/simple flashinfer-python
python demo.py --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/university \
    --backend flashinfer

```

The `FlashInferKVCacheManager` in [`lingbot_map/layers/flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/flashinfer_cache.py) builds a **two-stream paged layout** separating patch pages from special tokens. By recycling patch pages and never evicting the special token stream, the manager keeps the visible page table compact (approximately 88 pages for a 64-frame window) and reuses the same plan for all layers, eliminating per-layer kernel launches.

## Reduce the KV-Cache Window with Keyframe Intervals

The cache size grows linearly with the sliding window (`sliding_window` parameter defaults to 64). For sequences longer than the training range (~320 frames), the KV-cache becomes a memory bottleneck.

Control the cache growth using these flags:

- **`--keyframe_interval N`** – Retain only every *N*-th frame in the KV-cache. Non-keyframes still produce predictions but do not consume cache slots.
- **`--mode windowed --window_size W`** – Reset the cache every *W* slots, enabling arbitrarily long runs.
- **`--overlap_keyframes 8`** – Share context between windows to maintain temporal coherence.

Run a 10,000-frame video with 2-frame keyframes:

```bash
python demo.py \
    --image_folder path/to/long_video_frames \
    --model_path /path/to/lingbot-map-long.pt \
    --keyframe_interval 2 \
    --mode windowed --window_size 128 \
    --overlap_keyframes 8

```

The `windowed` mode triggers `FlashInferKVCacheManager.evict_frames` only after the sliding window exceeds the configured limit, while `keyframe_interval` caps the number of cached frames, keeping memory roughly constant regardless of sequence length.

## Compile the Model for Target Hardware

LingBot-Map includes a JIT compilation helper in [`gct_profile.py`](https://github.com/Robbyant/lingbot-map/blob/main/gct_profile.py) that optimizes attention kernels for your specific CUDA architecture. Compiled models eliminate Python overhead and allow FlashInfer to run at peak speed.

Compile for your GPU:

```bash
python gct_profile.py --backend flashinfer --dtype bf16 --compile

```

The compiled model is automatically loaded by [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) on subsequent runs. The profiling script also reports per-layer runtime, helping identify remaining hotspots in [`lingbot_map/layers/attention.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/attention.py).

## Use Lower-Precision Data Types

When FlashInfer is active, the cache stores tensors in **bfloat16** (or fp16). You can force fp32 for debugging, but for best performance, use the default lower-precision mode:

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --dtype bf16

```

The `dtype` argument propagates to `FlashInferKVCacheManager` where `self.dtype` is set in the constructor, halving the memory bandwidth requirements compared to fp32.

## Offload Per-Frame Predictions to CPU

When GPU memory is constrained, the `--offload_to_cpu` flag moves dense point-cloud tensors off-GPU after each frame. This is enabled by default, but you can explicitly toggle it:

```bash

# Enable offloading (default, saves VRAM)

python demo.py --image_folder example/university --offload_to_cpu

# Disable offloading (if you have spare GPU memory)

python demo.py --image_folder example/university --no-offload_to_cpu

```

Disabling offloading can shave a few milliseconds per frame by eliminating CPU-GPU transfer overhead, but requires approximately 500MB additional VRAM for typical 518×378 inputs.

## Reduce Iterative Camera Refinement

The camera head in [`lingbot_map/layers/vision_transformer.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/vision_transformer.py) performs `camera_num_iterations` (default 4) refinement passes per frame. Each iteration triggers a full forward pass through the transformer, making this a linear cost multiplier.

Reduce iterations for maximum speed:

```bash
python demo.py \
    --image_folder example/university \
    --model_path /path/to/lingbot-map-long.pt \
    --camera_num_iterations 1

```

Setting `--camera_num_iterations 1` reduces transformer executions by 75% at a modest cost to pose accuracy, ideal for real-time preview processing.

## Pre-compute Sky Masks to Eliminate Segmentation Overhead

Sky masking adds a forward pass through an ONNX segmentation model. Cache these masks once and reuse them across runs:

```bash
python demo.py \
    --image_folder example/oxford \
    --mask_sky \
    --sky_mask_dir ./oxford_sky_masks

```

Masks are saved under `<image_folder>_sky_masks/` and loaded instantly on subsequent runs, bypassing the segmentation model entirely.

## Profile and Tune with gct_profile.py

Use the profiling script to measure FPS across different resolutions, window sizes, and backends:

```bash
python gct_profile.py \
    --backend flashinfer \
    --dtype bf16 \
    --keyframe_interval 2 \
    --window_size 128

```

The output includes a breakdown by module (attention, KV-cache, camera head), enabling data-driven decisions when you optimize performance in LingBot-Map for your specific hardware.

## Summary

- **Install FlashInfer** (`pip install flashinfer-python`) and use `--backend flashinfer` to enable paged KV-cache management that eliminates per-layer kernel launches.
- **Reduce cache pressure** with `--keyframe-interval 2` and `--mode windowed --window_size 128` to keep memory under 2 GB for 10,000+ frame sequences.
- **Compile the model** using `gct_profile.py --compile` to optimize attention kernels for your CUDA architecture.
- **Lower precision** to `bf16` to halve memory bandwidth usage.
- **Minimize camera iterations** to 1 for maximum throughput when pose accuracy is less critical than speed.
- **Cache sky masks** offline to avoid repeated ONNX segmentation inference.

## Frequently Asked Questions

### What is the fastest backend for LingBot-Map inference?

**FlashInfer** is the fastest backend. According to the source code in [`lingbot_map/layers/flashinfer_cache.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/flashinfer_cache.py), FlashInfer implements a two-stream paged KV-cache that avoids copying the entire cache each step, while the fallback PyTorch SDPA requires expensive memory copies on long sequences. Use `--backend flashinfer` after installing `flashinfer-python`.

### How do I run LingBot-Map on videos longer than 10,000 frames without running out of memory?

Use **windowed mode** with a keyframe interval. Combine `--mode windowed --window_size 128` with `--keyframe-interval 2` to reset the KV-cache periodically while retaining only every second frame in the cache. This keeps GPU memory roughly constant regardless of video length, as implemented in `FlashInferKVCacheManager.evict_frames`.

### Why is my GPU memory usage so high during inference?

High memory usage typically stems from the KV-cache growing with sequence length or camera refinement iterations. Reduce `--camera_num_iterations` from 4 to 1 to eliminate three full transformer passes per frame. Additionally, ensure `--offload_to_cpu` is enabled (default) to move point-cloud tensors off the GPU immediately after generation.

### Does lowering camera iterations affect reconstruction accuracy?

Yes, reducing `--camera_num_iterations` from the default 4 to 1 will modestly reduce pose accuracy because the camera head performs iterative refinement. However, the first iteration already provides a reasonable estimate, and the speed improvement (4× fewer transformer passes) is often worth the trade-off for real-time applications or preview processing.