How to Visualize 3D Reconstructions from LingBot-Map: A Complete Guide

LingBot-Map provides an interactive web-based viewer built on Viser that launches a local server—typically at localhost:8080—to explore 3D point clouds, camera frustums, and reconstructed trajectories directly in your browser.

The visualization pipeline in the Robbyant/lingbot-map repository converts per-frame prediction dictionaries into interactive 3D scenes. Whether you are processing a single image folder or streaming a video, the system handles depth unprojection, camera geometry, and optional sky masking to produce clean, navigable reconstructions.

Understanding the Visualization Pipeline

The visualization workflow follows a clear path from model inference to interactive rendering. The primary components are orchestrated through demo.py, which feeds processed predictions into the PointCloudViewer class.

Model Inference and Post-Processing

Visualization begins after the model generates predictions. In demo.py, the script loads input media via load_images and runs the streaming or windowed model (GCTStream). The postprocess function—located at line 78 in demo.py—converts raw pose encodings into camera extrinsics and intrinsics, producing a prediction dictionary containing depth maps, confidence scores, and camera parameters.

This dictionary serves as the input for the visualization layer, passing through _process_pred_dict in the viewer to generate world-point maps via unproject_depth_map_to_point_map.

The PointCloudViewer Architecture

The core rendering engine lives in lingbot_map/vis/point_cloud_viewer.py. The PointCloudViewer class initializes a Viser server on a user-specified port (defaulting to 8080) and provides:

• Point cloud generation from depth unprojection
• Camera frustum visualization showing pose and field of view
• Trajectory rendering as a continuous path through the scene
• Real-time GUI controls for point size, down-sampling, confidence thresholds, and 3D/4D playback
• Screenshot and GLB export functionality

The viewer automatically handles the heavy lifting of converting 2.5D depth predictions into metric 3D point clouds while maintaining camera geometry consistency.

Optional Sky Masking

For outdoor scenes, you can activate automatic sky removal. When the --mask_sky flag is provided, the viewer loads an ONNX segmentation model from HuggingFace via apply_sky_segmentation in lingbot_map/vis/sky_segmentation.py. This filters sky points before they enter the point cloud, reducing noise and improving visual clarity.

Interactive Visualization Methods

LingBot-Map supports multiple input modalities for immediate visual feedback. All methods launch the same Viser-based interface.

Visualizing Local Image Folders

To explore a static scene from a collection of images:

# Install visualization dependencies

pip install -e ".[vis]"

# Run the interactive demo

python demo.py \
  --model_path /path/to/lingbot-map-long.pt \
  --image_folder example/university \
  --mask_sky

Open http://localhost:8080 to view the reconstruction. The GUI panel allows you to adjust point size, apply down-sampling, and threshold confidence values in real-time.

Streaming from Video Files

For temporal sequences, process video files with custom frame rates:

python demo.py \
  --model_path /path/to/checkpoint.pt \
  --video_path my_scene.mp4 \
  --fps 10 \
  --port 9000

The --port flag allows you to run multiple viewers simultaneously on different ports. The streaming mode maintains temporal consistency in the visualization, showing how the camera trajectory evolves through the scene.

Offline and Export Options

Beyond interactive exploration, LingBot-Map supports batch rendering and standard 3D format exports.

Batch Rendering for Video Production

For creating sharable MP4 fly-throughs without manual interaction, use the offline pipeline in demo_render/batch_demo.py:

python demo_render/batch_demo.py \
  --video_path my_long_scene.mp4 \
  --output_folder outputs/long_scene \
  --model_path /path/to/lingbot-map.pt \
  --config demo_render/config/indoor.yaml \
  --mode windowed \
  --window_size 128 \
  --keyframe_interval 13 \
  --sky_mask_dir outputs/sky_masks \
  --camera_vis default \
  --keyframes_only_points \
  --frame_tag --frame_tag_position top_right \
  --save_predictions

This script reuses the same PointCloudViewer code path via the save_video method, ensuring the output matches the interactive view exactly.

Exporting to GLB Format

To import reconstructions into Blender, Unity, or other 3D software, export to GLB:

# Interactive GUI method: Open the "Export GLB" folder in the browser and click "Export GLB"

# Programmatic method:

python -c "
from lingbot_map.vis.point_cloud_viewer import PointCloudViewer

# Assume pred_dict obtained from demo.py post-processing

viewer = PointCloudViewer(pred_dict=pred_dict, port=8081)
viewer.glb_output_path.value = 'scene.glb'
viewer._export_glb()
"

The exported .glb file contains the filtered point cloud, camera frustum meshes, and an optional trajectory tube, preserving all visual filters applied in the GUI.

Summary

• LingBot-Map visualizes 3D reconstructions using a Viser-based web viewer (PointCloudViewer) that runs locally on port 8080 by default.
• The pipeline flows from demo.py inference through postprocess to generate prediction dictionaries, which _process_pred_dict converts into point clouds and camera geometry.
• Interactive features include real-time point size adjustment, confidence thresholding, sky masking, and 3D/4D playback controls.
• Offline rendering via demo_render/batch_demo.py produces MP4 videos using the same rendering code as the interactive viewer.
• GLB export allows you to extract scenes for external 3D software, including point clouds and camera trajectories.

Frequently Asked Questions

How do I launch the viewer on a different port?

Pass the --port argument to demo.py. For example, --port 9000 starts the server on localhost:9000. This is useful when running multiple reconstructions simultaneously or when port 8080 is occupied.

What is the difference between interactive mode and batch rendering?

Interactive mode launches the Viser server and opens a browser window for real-time exploration with GUI controls. Batch rendering (demo_render/batch_demo.py) uses the same underlying PointCloudViewer class but runs headlessly to output MP4 videos, making it suitable for processing long sequences or generating presentation assets without manual intervention.

Can I remove sky points from outdoor reconstructions?

Yes. Add the --mask_sky flag when running demo.py. This triggers apply_sky_segmentation from lingbot_map/vis/sky_segmentation.py, which loads an ONNX model to identify and filter sky regions before they appear in the point cloud. This significantly reduces floating artifacts in outdoor scenes.

Which file contains the main visualization logic?

The core implementation resides in lingbot_map/vis/point_cloud_viewer.py. This file contains the PointCloudViewer class, which handles Viser server initialization, point cloud creation via unproject_depth_map_to_point_map, camera frustum rendering, and GLB export functionality.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}