How to Set Up Robbyant/lingbot-map Locally: Complete Installation Guide

To set up Robbyant/lingbot-map locally, clone the repository, install dependencies via Poetry, compile the CUDA kernels using NVCC, download sample datasets with the provided script, and run demo.py to verify the installation.

LingBot-Map is a Python-based framework for creating dense 3-D maps from RGB-D video streams. According to the Robbyant/lingbot-map source code, the project relies on Poetry for dependency management and requires CUDA-capable hardware for GPU-accelerated point cloud processing. This guide provides the exact commands and file paths needed to build a working local environment.

Prerequisites and System Requirements

Hardware Requirements

• NVIDIA GPU with CUDA compute capability
• NVCC compiler installed and available in your system PATH
• Sufficient disk space for sample datasets (Oxford, TUM, KITTI)

Software Dependencies

• Python 3.8 or higher
• Poetry (Python packaging and dependency management)
• Matching CUDA toolkit version for your NVIDIA driver
• PyTorch installation compatible with your CUDA version (verify with torch.cuda.is_available())

Step-by-Step Local Installation

1. Clone the Repository

Pull the source code and navigate to the project directory:

git clone https://github.com/Robbyant/lingbot-map.git && cd lingbot-map

2. Install Python Dependencies

The project uses Poetry (pyproject.toml) to declare exact versions of required packages including numpy, torch, open3d, and torchvision:

python -m pip install poetry && poetry install

3. Compile CUDA Kernels for GPU Acceleration

The pre-processing pipelines contain CUDA kernels for point-visibility computation (preprocess/points_visibility/visibility_kernel.cu) and frustum culling (preprocess/points_visibility/frustum_cull.cu). While these compile automatically on first import, manual compilation accelerates development:

python -c "import lingbot_map.vis.utils as u; u.build_cuda()"

If the automatic build fails, compile manually:

nvcc -O3 -shared -o visibility_kernel.so preprocess/points_visibility/visibility_kernel.cu

4. Download Sample Datasets

Prepare the benchmark environments by downloading Oxford Spires, TUM, and KITTI sequences:

bash benchmark/envs/install_all.sh

5. Run the Demo

Verify your installation by processing the example sequence in example/university:

python demo.py --config demo_render/config/default.yaml

This launches the interactive viewer and confirms that the Aggregator Core and Visualization Suite are functioning correctly.

Core Architecture Components

Understanding the three main components helps troubleshoot setup issues:

Pre-processing Pipelines

Located in preprocess/points_visibility/, these CUDA kernels generate per-frame visibility masks. The visibility_kernel.cu and frustum_cull.cu files accelerate point cloud filtering before map aggregation.

Aggregator Core

The lingbot_map/aggregator package handles map construction:

• lingbot_map/aggregator/base.py defines the abstract BaseAggregator class
• lingbot_map/aggregator/stream.py implements StreamAggregator, the concrete streaming pipeline used for incremental mapping

Visualization Suite

Found in lingbot_map/vis/, this module provides:

• point_cloud_viewer.py – Interactive Open3D viewer
• glb_export.py – GLB format export for external 3-D tools
• sky_segmentation.py – Optional sky removal preprocessing

Working with the Framework

Importing and Processing Video Streams

Once set up locally, initialize the aggregator and process RGB-D frames:

from lingbot_map.aggregator.stream import StreamAggregator
from lingbot_map.vis.point_cloud_viewer import PointCloudViewer

# Initialise the aggregator with 0.05 m voxel size

agg = StreamAggregator(voxel_size=0.05)

# Process frames from your RGB-D source

for rgb, depth in my_video_source():
    agg.process_frame(rgb, depth)

# Visualise the resulting map

viewer = PointCloudViewer()
viewer.show(agg.get_global_point_cloud())

Exporting Maps to GLB

Export dense maps for use in Blender or other 3-D software:

from lingbot_map.vis.glb_export import export_glb

pc = agg.get_global_point_cloud()
export_glb(pc, "my_map.glb")

Running Benchmarks

To reproduce the paper's quantitative results after setting up locally:

python benchmark/run.py --dataset oxford_spires

Results are stored under benchmark/report/.

Summary

• Clone the repository from https://github.com/Robbyant/lingbot-map.git
• Install dependencies using poetry install as defined in pyproject.toml
• Compile CUDA kernels via lingbot_map.vis.utils.build_cuda() or manual NVCC invocation
• Download sample data with benchmark/envs/install_all.sh
• Verify functionality by running demo.py with the default configuration
• Process RGB-D streams using StreamAggregator from lingbot_map/aggregator/stream.py

Frequently Asked Questions

Do I need an NVIDIA GPU to set up Robbyant/lingbot-map locally?

Yes, the framework requires CUDA-capable hardware. The pre-processing pipelines in preprocess/points_visibility/visibility_kernel.cu and frustum_cull.cu compile to GPU-specific binaries and accelerate point-visibility calculations essential for real-time mapping.

How do I fix CUDA kernel compilation errors?

Ensure your PyTorch installation matches your system's CUDA driver version by verifying torch.cuda.is_available() returns True. If automatic compilation fails during import, manually compile the kernels using nvcc -O3 -shared -o visibility_kernel.so visibility_kernel.cu from the repository root.

What datasets are compatible with the local setup?

The repository includes automated download scripts for Oxford Spires, TUM RGB-D, and KITTI odometry sequences. Run bash benchmark/envs/install_all.sh to acquire these datasets, or adapt the StreamAggregator class to accept custom RGB-D video streams.

Can I install dependencies without using Poetry?

While pyproject.toml specifies the canonical environment, you can manually install the core dependencies (numpy, torch, open3d, torchvision) via pip. However, Poetry is strongly recommended to ensure version compatibility between the CUDA kernels and the Python scientific stack.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}