# What Is LingBot‑Map Used For? Streaming 3D Reconstruction from Monocular Video

> Discover LingBot-Map, a real-time streaming 3D reconstruction model from monocular video. Achieve 3D mapping at 20 FPS for over 10,000 frames.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: getting-started
- Published: 2026-07-19

---

**LingBot‑Map is a feed‑forward 3D foundation model designed for real‑time streaming 3D reconstruction from monocular image sequences, processing over 10,000 frames at 20 FPS using a geometric context transformer.**

The Robbyant/lingbot‑map repository provides an open‑source implementation of a **streaming 3D reconstruction** system that eliminates the need for iterative optimization. Unlike traditional SLAM or NeRF pipelines, LingBot‑Map processes RGB video frames in a single forward pass while maintaining geometric consistency across thousands of frames through specialized memory mechanisms.

## Core Architecture and Capabilities

### Geometric Context Transformation

The model centers on a **Geometric Context Transformer** implemented in [`lingbot_map/layers/vision_transformer.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/vision_transformer.py). This architecture extends the Vision Transformer (ViT) design with custom blocks for geometric token handling. It simultaneously grounds image coordinates, integrates dense geometric cues including depth and sky masks, and corrects long‑range drift through three memory mechanisms: an **anchor context**, a **pose‑reference window**, and **trajectory memory**.

### High‑Efficiency Streaming Inference

LingBot‑Map achieves **20 FPS** inference speed on 518×378 resolution images using **paged KV‑cache attention** via FlashInfer. The system maintains this performance even for sequences longer than **10,000 frames**, making it suitable for robotics and AR/VR applications requiring real‑time processing.

### State‑of‑the‑Art Benchmark Performance

According to the evaluation logic in [`benchmark/run.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/run.py), the model outperforms both streaming baselines and classic iterative SLAM/NeRF pipelines on standard datasets including **KITTI**, **Oxford Spires**, and indoor long‑video demonstrations.

## Running LingBot‑Map: Interactive and Batch Modes

### Launching the Interactive Viewer

The [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) script launches a browser‑based Viser viewer for interactive exploration. Run the following command to start reconstruction on an image sequence:

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/university \
    --mask_sky

```

This starts the web interface at `http://localhost:8080` using the visualization wrapper defined in [`lingbot_map/vis/viser_wrapper.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py).

### Configuring Keyframe Intervals

To reduce memory usage while preserving drift correction, process only every N‑th frame as a keyframe:

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/loop \
    --keyframe_interval 5 \
    --mask_sky

```

### Processing Long Sequences with Windowed Inference

For sequences exceeding 3,000 frames, enable sliding‑window attention to manage memory:

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/long_video \
    --window_size 3000 \
    --mask_sky

```

### Offline Batch Rendering

Use [`demo_render/batch_demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/batch_demo.py) for rendering very long sequences (e.g., 25,000‑frame walkthroughs) to video or GLB format without interactive latency:

```bash
python demo_render/batch_demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/indoor_long \
    --out_dir renders/indoor_long \
    --window_size 4000 \
    --mask_sky

```

## Key Implementation Files

- **[`lingbot_map/layers/vision_transformer.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/vision_transformer.py)** – Implements the Geometric Context Transformer with geometric token handling, memory‑efficient attention, and optional sky‑masking logic.
- **[`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py)** – Interactive demo entry point that launches the Viser viewer and handles real‑time streaming inference.
- **[`demo_render/batch_demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/batch_demo.py)** – Offline rendering pipeline for batch processing very long video sequences without interactive overhead.
- **[`benchmark/run.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/run.py)** – Evaluation driver for testing reconstruction quality on KITTI, Oxford Spires, and other standard benchmarks.
- **[`lingbot_map/vis/viser_wrapper.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py)** – Wrapper around the Viser visualization library used by the interactive demo.

## Summary

- LingBot‑Map performs **streaming 3D reconstruction** from monocular video without iterative optimization or offline training.
- The **Geometric Context Transformer** in [`lingbot_map/layers/vision_transformer.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/vision_transformer.py) integrates geometric cues and corrects drift via anchor contexts and trajectory memory.
- Achieves **20 FPS** on 518×378 images for sequences over **10,000 frames** using FlashInfer paged attention.
- Supports both **interactive visualization** via [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) and **offline batch rendering** via [`demo_render/batch_demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/batch_demo.py).
- Outperforms SLAM and NeRF baselines on KITTI and Oxford Spires benchmarks according to [`benchmark/run.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/run.py).

## Frequently Asked Questions

### What hardware is required to run LingBot‑Map?

LingBot‑Map requires a GPU capable of accelerating FlashInfer‑based attention mechanisms. The system maintains real‑time performance at 20 FPS on modern GPUs while processing 518×378 resolution frames, and scales to sequences exceeding 10,000 frames through efficient KV‑cache management.

### How does LingBot‑Map correct drift over long trajectories?

The architecture employs three complementary mechanisms within the Geometric Context Transformer: an **anchor context** for global reference, a **pose‑reference window** for local consistency, and **trajectory memory** for temporal integration. These components allow the model to maintain geometric accuracy without explicit loop closure optimization.

### Can LingBot‑Map process RGB‑D input or only monocular video?

While primarily designed for **monocular image sequences**, the model architecture supports integration of dense geometric cues such as predicted depth and sky masks. The feed‑forward pipeline operates on standard RGB video, though depth predictions can enhance reconstruction quality when available.

### Where is the core transformer architecture implemented?

The **Geometric Context Transformer** is implemented in [`lingbot_map/layers/vision_transformer.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/layers/vision_transformer.py). This file contains the custom transformer blocks that handle geometric tokens, manage the paged KV‑cache attention via FlashInfer, and process optional sky‑masking inputs for outdoor scene reconstruction.