How to Use LingBot-Map with SDPA Fallback: Complete Configuration Guide

To use LingBot-Map with SDPA fallback, add the --use_sdpa flag to your demo.py command, which automatically switches from FlashInfer to PyTorch's Scaled Dot-Product Attention and creates a dict-based KV cache without requiring additional dependencies.

The LingBot-Map repository by Robbyant provides a streaming 3D reconstruction system that supports multiple attention backends. When the optimized FlashInfer kernels are unavailable due to CUDA version incompatibility or installation constraints, you can activate the LingBot-Map SDPA fallback to run inference using PyTorch's native attention implementation.

Understanding the SDPA Fallback Architecture

FlashInfer vs. SDPA Backends

LingBot-Map implements two distinct attention mechanisms in lingbot_map/aggregator/stream.py. The FlashInfer backend (default) utilizes custom CUDA kernels with a paged KV-cache for optimal streaming performance. The SDPA (Scaled Dot-Product Attention) backend serves as a pure-Pytorch fallback that requires no additional installations.

When to Use the SDPA Fallback

Choose the SDPA fallback when:

• FlashInfer cannot be compiled or installed (incompatible CUDA/PyTorch versions)
• Running on systems without FlashInfer wheels available
• Debugging or developing in environments where simplified dependencies are preferred

According to the source code, SDPA produces identical reconstruction quality but operates at approximately 5-10 FPS on an RTX 3080 compared to FlashInfer's 20 FPS.

How to Enable SDPA Fallback in LingBot-Map

Command-Line Flag Configuration

The entry point in demo.py defines the --use_sdpa argument at lines 82-84. When provided, this flag sets args.use_sdpa = True, which propagates through the model initialization chain.

Model Construction and Aggregator Selection

In demo.py lines 48-49, the load_model function forwards the use_sdpa parameter to the model constructor. The AggregatorStream.__init__ method in lingbot_map/aggregator/stream.py (lines 85-94) detects this flag via kwargs.pop('use_sdpa', False) and stores it as self.use_sdpa.

The backend selection occurs at lines 136-138 of the same file, where the code conditionally chooses between SDPABlock and FlashInferBlock based on this boolean flag.

KV-Cache Implementation

When SDPA is active, the aggregator creates a lightweight Python dictionary (self.kv_cache) for each block at lines 86-94 of aggregator/stream.py. This contrasts with FlashInfer's paged KV-cache manager, which requires the flashinfer-python dependency.

Running LingBot-Map with SDPA: Code Examples

Basic SDPA Inference

To run the interactive demo without FlashInfer:

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder /data/example/courthouse \
    --mask_sky \
    --use_sdpa

Windowed Mode with Keyframe Reduction

For long sequences, combine SDPA with windowed processing:

python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --video_path /data/long_video.mp4 \
    --fps 10 \
    --mode windowed \
    --window_size 128 \
    --keyframe_interval 2 \
    --use_sdpa

Verifying Backend Selection Programmatically

Inspect the model after loading to confirm SDPA activation:

from lingbot_map.models.gct_stream import load_model

model = load_model(args, device)
print("Using SDPA:", model.aggregator.use_sdpa)  # Returns True if --use_sdpa was passed

Performance Characteristics

The SDPA fallback trades speed for compatibility. While FlashInfer achieves approximately 20 FPS on an RTX 3080 using optimized CUDA kernels, the SDPA backend operates at 5-10 FPS using standard PyTorch operations. However, the reconstruction quality remains identical because both paths use the same model trunk and heads defined in lingbot_map/models/gct_stream.py.

Summary

• Add --use_sdpa to any demo.py command to activate the PyTorch attention backend
• The flag is parsed in demo.py (lines 82-84) and propagated through load_model (lines 48-49)
• AggregatorStream selects SDPABlock instead of FlashInferBlock when the flag is detected (lines 136-138 of aggregator/stream.py)
• SDPA uses a simple Python dict for KV caching (lines 86-94) rather than FlashInfer's paged memory manager
• Expect 2-4x slower inference compared to FlashInfer, with identical output quality

Frequently Asked Questions

Do I need to install FlashInfer to use LingBot-Map?

No. If you use the --use_sdpa flag, LingBot-Map operates entirely with PyTorch's built-in Scaled Dot-Product Attention and does not import or require the flashinfer-python package. This makes installation possible on systems where FlashInfer's CUDA kernels cannot be compiled.

Why is my SDPA inference slower than FlashInfer?

SDPA relies on PyTorch's generic matrix multiplication operations rather than FlashInfer's hand-optimized CUDA kernels for paged KV-cache management. According to benchmarks in the repository, SDPA achieves approximately 5-10 FPS on an RTX 3080 compared to FlashInfer's 20 FPS, representing the performance cost of the portable fallback implementation.

Can I use windowed mode and keyframe reduction with SDPA?

Yes. All inference modes including --mode windowed, --window_size, and --keyframe_interval function identically with the SDPA backend. The only difference is the underlying attention computation; the streaming aggregator logic remains the same regardless of which block class (SDPABlock or FlashInferBlock) is active.

How do I verify that SDPA is actually being used?

After loading the model, check the use_sdpa attribute on the aggregator instance: model.aggregator.use_sdpa will return True when the SDPA fallback is active. Additionally, if the code executes without importing flashinfer and without raising import errors, the PyTorch backend is successfully engaged.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}