How to Use viewer.py for 3D Visualization in LingBot-Map

Launch python -m benchmark.viewer <workspace> to start an interactive Viser web server that renders camera trajectories, point clouds, and RGB frames from LingBot-Map benchmark results.

The viewer.py module in the Robbyant/lingbot-map repository provides a browser-based interface for inspecting 3D reconstruction results. This tool enables researchers to visualize camera trajectories, per-frame point clouds, and RGB thumbnails without writing custom plotting code. Understanding how to use viewer.py for 3D visualization streamlines the debugging and presentation of SLAM and mapping benchmarks stored in the Benchmark Storage Structure (BSS) format.

Prerequisites and Installation

Installing Visualization Dependencies

To use the viewer, you must install the optional visualization dependencies. This installs viser>=0.2.23, trimesh, matplotlib, onnxruntime, and requests.

pip install "lingbot-map[vis]"

Launching the 3D Viewer

The viewer requires a workspace directory containing BSS entries marked with .complete.json files. These markers indicate completed benchmark runs.

Basic CLI Usage

Serve the current directory as the workspace:

python -m benchmark.viewer .

Command-Line Arguments

The argparse block at lines 63–82 of benchmark/viewer.py defines the following arguments:

python -m benchmark.viewer ./my_workspace -p 8081 -t 5 -s 4 --verbose

• workspace – Root folder of benchmark results (default: .)
• -p/--port – HTTP port for the Viser server (default: 20540)
• -t/--temporal-subsample – Display every t-th frame (default: 1)
• -s/--spatial-subsample – Downsample point-cloud pixels by factor s (default: 2)
• --verbose – Enable debug logging

After launching, open your browser to http://<HOST_IP>:<PORT> (the server prints the host IP on startup).

Core Architecture and Data Pipeline

Understanding the internal architecture helps troubleshoot loading issues. The implementation in benchmark/viewer.py organizes functionality into several key components that handle workspace scanning, client sessions, and data caching.

Workspace Scanning and Global Context

The scan_workspace() function (lines 331–389) recursively walks the workspace, collects directories containing .complete.json markers, and builds a nested dictionary structure[dataset][scene] → [methods]. It guarantees that ground-truth (gt) entries appear first in the menu.

The GlobalContext class (lines 60–73) holds static configuration including the workspace path, available data flags, and default subsampling parameters. This context is shared across all client sessions via the module-level _global_context variable.

Client Session Management

Each browser connection instantiates a ClientClosures object (lines 444–506). This class creates the GUI dropdowns, sliders, and buttons, handles UI callbacks, and orchestrates heavy data loading via _rebuild_scene() (lines 1315–1495).

Scene Caching System

The SceneCache class (lines 83–155) persists trajectories, point clouds, RGB thumbnails, and sky masks to disk at workspace/viewer_cache/. Before reuse, it validates cache parameters—including frame count, image size, spatial subsample factor, and alignment—to ensure consistency with current settings.

Interactive Features and UI Controls

Once connected via browser, the interface provides several control panels for data exploration.

Data Selection and Scene Loading

The top folder contains dropdown menus for Dataset → Scene → Method selection. After clicking Load, the viewer executes _rebuild_scene(), which performs the following steps:

1. Loads BSS artifacts via BSSManager
2. Retrieves trajectories, intrinsics, and optional confidence masks
3. Generates or loads point clouds (per-frame grid or global fallback)
4. Computes base point sizes via _compute_base_point_size
5. Populates the Viser scene with camera frustums and geometry

Sampling and Filtering Controls

The Sampling folder adjusts temporal and spatial subsampling sliders to thin data on-the-fly. The Scene folder provides:

• Confidence Threshold – Filters points by confidence value (active only when the method provides depth_conf or world_points_conf tensors)
• Point Scale – Logarithmic scaling for point cloud rendering
• Remove Sky – Applies binary sky masks generated by sky_segmentation (lines 1241–1250) and cached after first computation
• Frustum Size – Adjusts the visual scale of camera trajectory markers

Playback and Navigation

The _rebuild_playback_gui() function (lines 1999–2021) adds animation controls including a timestep slider, framerate adjustment, play/pause button, and loop toggle. The Camera Clipboard feature allows copying viewpoints via the Copy button and pasting them into other browser sessions with Paste, enabling synchronized viewpoints across multiple clients.

Programmatic Usage Examples

Beyond the CLI, you can integrate the viewer into Python scripts or Jupyter notebooks.

Minimal Programmatic Launch

Call run_viser() directly (defined at lines 1894–1902) to start the server from code:

from pathlib import Path
from benchmark.viewer import run_viser

run_viser(
    workspace=Path("/tmp/lingbot_workspace"),
    port=8080,
    temporal_subsample=2,
    spatial_subsample=4,
    verbose=True,
)

Loading BSS Entries Manually

For custom analysis, access the underlying loaders directly as implemented in _rebuild_scene() (lines 1444–1453):

from benchmark.core.loader import BSSLoader
from benchmark.core.storage import BSSManager
from pathlib import Path

workspace = Path("./workspace")
manager = BSSManager(workspace)

# Load ground truth: use None for method parameter

artifact = manager.get_artifact("my_dataset", "scene_01", None)
loader = BSSLoader(artifact)

traj = loader.load_trajectory()        # Returns (N, 4, 4) camera poses

rgb = loader.load_rgb_list()           # Returns list of (H, W, 3) uint8 images

pc = loader.load_point_cloud_grid()    # Returns (V, H, W, 6) xyzrgb+mask array

Stand-Alone Point Cloud Viewer

For visualizing arbitrary point clouds without the BSS infrastructure, use PointCloudViewer from lingbot_map/vis/point_cloud_viewer.py (lines 70–96):

from lingbot_map.vis.point_cloud_viewer import PointCloudViewer

# pcs, colors, confidences are lists of per-frame numpy arrays

viewer = PointCloudViewer(
    pc_list=pcs,
    color_list=colors,
    conf_list=confidences,
    cam_dict=my_cam_dict,
    port=8082,
    show_camera=True,
    mask_sky=True,
    image_folder="./images",
)

Cache Management and Performance Optimization

The first load of a scene triggers computation of point clouds, trajectory geometry, and sky masks, which can be slow. The SceneCache class writes these to workspace/viewer_cache/ for near-instant reloads on subsequent visits.

If you modify underlying BSS data (e.g., re-running a method), you must invalidate the cache. Click the Clear Cache button in the UI, or manually delete the viewer_cache folder. The cache automatically validates parameters (frame count, image size, subsample factors) when loading, but manual clearing is required when source data changes.

Summary

• Installation – Use pip install "lingbot-map[vis]" to obtain Viser and dependencies
• Launch – Execute python -m benchmark.viewer <workspace> with optional -p, -t, and -s flags
• Architecture – run_viser() initializes the server, ClientClosures handles per-client UI, and SceneCache manages disk persistence at workspace/viewer_cache/
• Features – Interactive controls include confidence filtering, sky removal via sky_segmentation, temporal subsampling, and camera trajectory playback
• Programmatic API – Import run_viser() or PointCloudViewer for notebook-based visualization
• Performance – Disk caching accelerates reloads but requires manual invalidation after data updates

Frequently Asked Questions

What port does viewer.py use by default?

The Viser server binds to 0.0.0.0 using port 20540 by default. Specify a custom port using the -p or --port argument if this port is unavailable.

Why is the first load of a scene slow?

Initial loading requires computing camera frustums, generating point clouds from BSS data, and running sky segmentation if enabled. These results are cached to disk according to the SceneCache implementation (lines 83–155), making subsequent loads instantaneous.

How do I enable sky removal in the visualization?

Check the Remove Sky option in the Scene controls panel. This feature uses binary masks generated by sky_segmentation (lines 1241–1250). Masks are computed once and cached; use Clear Cache to regenerate them if needed.

Can I visualize results without the full benchmark infrastructure?

Yes. Import PointCloudViewer from lingbot_map/vis/point_cloud_viewer.py (lines 70–96) to render arbitrary point cloud arrays and camera dictionaries without requiring BSS directory structures or .complete.json markers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}