# How to Force LingBot-Map to Use the SDPA Backend Instead of FlashInfer

> Learn how to force LingBot-Map to use the SDPA backend instead of FlashInfer by setting use_sdpa=True or using the --use_sdpa flag for optimized performance.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-06

---

**Set `use_sdpa=True` when instantiating the model or pass the `--use_sdpa` command-line flag to switch from FlashInfer to PyTorch's native scaled-dot-product attention implementation.**

LingBot-Map (from the Robbyant/lingbot-map repository) provides two distinct attention backends for streaming inference: the default **FlashInfer** library with paged KV-cache support, and the **SDPA** (Scaled-Dot-Product Attention) backend using native PyTorch operations. This guide details exactly how to force LingBot-Map to use the SDPA backend instead of FlashInfer, removing the FlashInfer dependency while preserving inference capabilities.

## Understanding the Backend Architecture

LingBot-Map's streaming aggregator selects attention implementations based on the `use_sdpa` boolean flag. By default, `use_sdpa=False` enables **FlashInferBlock** and the `FlashInferKVCacheManager` for optimized memory management. When you force SDPA mode, the system instantiates **SDPABlock** classes and utilizes a simple Python dictionary-based KV cache instead.

The backend selection logic resides in [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py), where the constructor processes the `use_sdpa` keyword argument (lines 88-94):

```python
use_sdpa = kwargs.pop('use_sdpa', False)       # default value

self.use_sdpa = use_sdpa
self.use_flashinfer = not use_sdpa              # FlashInfer disabled when use_sdpa=True

```

This flag propagates through [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py) (lines 220-225), which forwards the configuration to the aggregator during `_build_aggregator`.

## Method 1: Force SDPA via Python API

To programmatically force LingBot-Map to use the SDPA backend, pass `use_sdpa=True` when constructing your model instance. This approach works with any model class that accepts the flag, such as `GCTStream`:

```python
import torch
from lingbot_map.models.gct_stream import GCTStream

# Build model with SDPA backend forced

model = GCTStream(
    img_size=518,
    patch_size=14,
    embed_dim=768,
    patch_embed="dinov2_vitl14_reg",
    pretrained_path="path/to/weights.pt",
    enable_3d_rope=True,
    kv_cache_sliding_window=64,
    kv_cache_scale_frames=8,
    use_sdpa=True,                 # Forces SDPA backend

)

# Inference runs with native PyTorch attention

dummy_imgs = torch.randn(1, 4, 3, 518, 378).cuda()
output = model(dummy_imgs)

```

When `use_sdpa=True`, the aggregator's `GlobalBlockCls` selector (lines 36-38 in [`stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/stream.py)) instantiates `SDPABlock` rather than `FlashInferBlock`, and the `_init_kv_cache` method initializes a dict-based cache instead of the FlashInfer paged cache.

## Method 2: Force SDPA via Command-Line Interface

For CLI-based workflows using [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py), add the `--use_sdpa` flag to force the SDPA backend without modifying source code. The argument parser (lines 82-84 in [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py)) exposes this flag:

```python
parser.add_argument("--use_sdpa", action="store_true", default=False,
                    help="Use SDPA backend (no flashinfer needed). Default: FlashInfer")

```

Execute your inference script with the flag:

```bash
python demo.py \
    --model_path weights.pt \
    --image_folder ./my_dataset \
    --use_sdpa               # Forces SDPA backend

```

This sets `use_sdpa=True` internally when the model builder constructs the `AggregatorStream` instance.

## Verifying the Active Backend

Confirm which backend is running by examining the log output from [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py). The `_init_kv_cache` method emits distinct messages for each backend:

- **FlashInfer active**: `INFO:lingbot_map.aggregator.stream:FlashInfer KV cache will be lazily initialized on first forward`
- **SDPA active**: `INFO:lingbot_map.aggregator.stream:SDPA KV cache initialized with X blocks`

You can also verify programmatically by inspecting the block class type:

```python
from lingbot_map.models.gct_stream import GCTStream

model = GCTStream(..., use_sdpa=True)

# Check the global block class inside the aggregator

block_type = type(model.aggregator.global_blocks[0]).__name__
print(block_type)   # prints "SDPABlock" when SDPA is forced

```

## Technical Implementation Details

The backend switch affects three critical components:

1. **Cache Manager**: In [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py), `_init_kv_cache` creates a `dict` object for SDPA mode versus a `FlashInferKVCacheManager` for standard mode.

2. **Attention Block Selection**: The `GlobalBlockCls` variable selects between `SDPABlock` and `FlashInferBlock` (defined in [`lingbot_map/layers/attention.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/attention.py)) based on the `use_sdpa` flag.

3. **Model Wrapper**: The `GCTStream` class in [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py) explicitly passes `use_flashinfer=not self.use_sdpa` and `use_sdpa=self.use_sdpa` to ensure consistency across the architecture.

## Summary

- **Force via Python**: Pass `use_sdpa=True` to `GCTStream` or any compatible model constructor.
- **Force via CLI**: Append `--use_sdpa` when running [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py).
- **Default behavior**: FlashInfer is used when `use_sdpa` is omitted or set to `False`.
- **Verification**: Check logs for "SDPA KV cache initialized" or inspect `model.aggregator.global_blocks[0]` for `SDPABlock` type.
- **Key files**: Configuration flows through [`lingbot_map/models/gct_stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/models/gct_stream.py) into [`lingbot_map/aggregator/stream.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/aggregator/stream.py), with CLI support in [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py).

## Frequently Asked Questions

### What is the difference between FlashInfer and SDPA backends?

FlashInfer utilizes the specialized FlashInfer library with optimized CUDA kernels and paged KV-cache management for maximum throughput. SDPA uses PyTorch's native `torch.nn.functional.scaled_dot_product_attention` with a simpler dictionary-based cache, requiring no additional dependencies beyond standard PyTorch.

### When should I force SDPA instead of using FlashInfer?

Force SDPA when FlashInfer installation fails, when running on hardware without CUDA support, or when debugging attention mechanisms where FlashInfer's optimized kernels obscure intermediate values. SDPA provides broader compatibility at the cost of reduced memory efficiency.

### How do I verify which backend is currently active?

Check the log output for initialization messages: SDPA mode logs "SDPA KV cache initialized with X blocks" while FlashInfer logs "FlashInfer KV cache will be lazily initialized". Alternatively, inspect `type(model.aggregator.global_blocks[0])` which returns `SDPABlock` for SDPA mode and `FlashInferBlock` for FlashInfer mode.

### Does using SDPA affect model accuracy or output quality?

No, both backends implement mathematically identical scaled-dot-product attention computations. The SDPA backend produces identical results to FlashInfer, differing only in implementation details and memory management strategies.