How to Configure the Batch Rendering Pipeline in LingBot-Map for Large-Scale Outdoor Scenes

To configure the batch rendering pipeline for large-scale outdoor scenes in LingBot-Map, use --mode windowed with a --window_size of 128 and --keyframe_interval of 13 to bound the KV-cache, while enabling --mask_sky to filter sky pixels and setting --point_size between 0.2 and 0.5 for sparse point visibility.

LingBot-Map ships with an offline rendering pipeline in demo_render/batch_demo.py that converts long outdoor videos into headless point-cloud fly-throughs. For sequences spanning thousands of frames—common in kilometre-scale outdoor reconstruction—you must tune the KV-cache windowing and sky segmentation parameters to prevent memory overflow and ensure clean visualizations. This guide details the precise flags, YAML configurations, and source code locations required to optimize the pipeline for outdoor environments according to the Robbyant/lingbot-map repository.

Understanding the Windowed KV-Cache for Long Sequences

Why Windowed Mode is Required

The default streaming KV-cache in LingBot-Map can only retain approximately 320 frames due to the RoPE (Rotary Position Embedding) training range constraints. For longer outdoor sequences, you must switch to windowed mode using --mode windowed, which implements a sliding window approach in the run_inference function (lines 779–805 in demo_render/batch_demo.py).

Calculating Window Coverage and Overlap

The effective coverage per window depends on three interacting parameters that are combined in the inference logic (lines 889–904):

• --window_size N: Total KV-cache slots per window (recommended 128).
• --keyframe_interval K: Cache only every K-th frame (recommended 13 for outdoor).
• --overlap_keyframes O: Shared keyframes between adjacent windows (recommended 8).

The first num_scale_frames slots (default 8) hold warm-up scale frames. The remaining slots hold keyframes, giving effective coverage of 8 + (window_size - 8) * keyframe_interval frames per window. With a window size of 128 and interval of 13, each window covers approximately 1,568 frames. Overlap is calculated internally as max(num_scale_frames, overlap_keyframes * keyframe_interval) to ensure smooth pose alignment across window boundaries.

Essential Configuration Parameters

The following flags interact to create a memory-bounded pipeline suitable for large outdoor scenes:

Flag	Purpose	Outdoor Recommendation
--mode windowed	Enables sliding KV-cache window	Required for sequences >320 frames
--window_size 128	Cache slots per window	128 (covers ~1,568 frames with interval 13)
--keyframe_interval 13	Frame sampling rate	13 (trade-off between density and memory)
--overlap_keyframes 8	Shared frames between windows	8 (ensures pose continuity)
--mask_sky	Enables sky segmentation	Required for clean outdoor geometry
--point_size 0.3	Rendered point diameter	0.2–0.5 (larger values for sparse scenes)
--keyframes_only_points	Restrict unprojection to keyframes	Reduces point-cloud size for long sequences
--config demo_render/config/outdoor_large.yaml	YAML preset with camera paths	Pre-configured for wide-angle outdoor footage

Enabling Sky Masking for Outdoor Scenes

ONNX Segmentation and Caching

Outdoor scenes require masking sky pixels to prevent infinite-depth artifacts. Enable the ONNX-based sky segmentation by adding --mask_sky, which triggers the visualize_sky_masks function (lines 778–819). The model (skyseg.onnx) downloads automatically on first use. Computed masks are cached to the directory specified by --sky_mask_dir, making subsequent runs instantaneous for the segmentation step.

Artifact Directory Configuration

The get_sky_artifact_dirs function (lines 658–674) propagates mask paths into the render pipeline configuration (lines 1178–1185). Specify --sky_mask_dir for binary masks and --sky_mask_visualization_dir for debug overlays to avoid recomputing segmentation on every render attempt.

Step-by-Step Configuration

1. Select the Outdoor Preset Use the provided YAML configuration that pre-fills camera paths and rendering defaults for wide-angle footage:

   --config demo_render/config/outdoor_large.yaml

2. Configure Windowed Inference For a 10,000-frame outdoor drive, set the window parameters to keep GPU memory bounded:

   --mode windowed \
   --window_size 128 \
   --keyframe_interval 13 \
   --overlap_keyframes 8

3. Enable Sky Masking and Rendering Filter sky pixels and adjust point size for sparse outdoor geometry:

   --mask_sky \
   --sky_mask_dir /data/outputs/drive/sky_masks \
   --sky_mask_visualization_dir /data/outputs/drive/sky_viz \
   --point_size 0.3 \
   --keyframes_only_points

4. Execute the Pipeline The process_scene function orchestrates the workflow: loading frames (lines 172–200), running windowed inference, saving NPZ predictions (save_predictions_npz, lines 1294–1376), and final video rendering (render_with_pipeline, lines 1488–1562).

Complete Command Examples

Basic Outdoor Batch Render

python demo_render/batch_demo.py \
    --video_path data/outdoor/driving.mp4 \
    --output_folder data/outdoor/render/ \
    --model_path models/lingbot-map-long.pt \
    --config demo_render/config/outdoor_large.yaml \
    --mode windowed \
    --window_size 128 \
    --keyframe_interval 13 \
    --overlap_keyframes 8 \
    --mask_sky \
    --point_size 0.4 \
    --keyframes_only_points \
    --save_predictions

Re-Render Existing NPZ with New Camera Path

python demo_render/batch_demo.py \
    --load_predictions data/outdoor/render/driving.npz \
    --output_folder data/outdoor/render/ \
    --config demo_render/config/outdoor_large.yaml \
    --camera_mode birdeye \
    --reveal_height_mult 3.0 \
    --point_size 0.2

Generate Sky Masks Only (Skip Inference)

python demo_render/batch_demo.py \
    --video_path data/outdoor/driving.mp4 \
    --output_folder data/outdoor/render/ \
    --mask_sky \
    --visualize_sky_mask_only

Key Source Files and Implementation Details

File	Role	Key Lines
demo_render/batch_demo.py	Main entry point for inference and rendering	172–200 (frame extraction), 779–805 (windowed mode), 889–904 (window calculation), 1072 (keyframes_only), 1110 (point_size), 1178–1185 (artifact dirs), 1294–1376 (NPZ saving), 1488–1562 (rendering)
demo_render/config/outdoor_large.yaml	Preset configuration for outdoor scenes	Camera paths and overlay defaults
lingbot_map/vis/sky_segmentation.py	ONNX sky segmentation and caching	Mask generation logic
rgbd_render/pipeline/builder.py	Voxel grid construction for rendering	Pipeline preparation
rgbd_render/config.py	YAML parsing and option storage	Used by render_with_pipeline

Summary

• Use --mode windowed once sequences exceed ~320 frames to enable sliding KV-cache windows in demo_render/batch_demo.py.
• Set --window_size 128, --keyframe_interval 13, and --overlap_keyframes 8 to cover approximately 1,500 frames per window while maintaining pose continuity via the overlap calculation.
• Enable --mask_sky and specify --sky_mask_dir to cache sky segmentation masks from the ONNX model, avoiding recomputation on subsequent runs.
• Adjust --point_size to 0.2–0.5 for sparse outdoor point clouds and use --keyframes_only_points to restrict unprojection to keyframes, reducing memory footprint.
• Leverage demo_render/config/outdoor_large.yaml as a base configuration for outdoor camera paths and rendering defaults.

Frequently Asked Questions

What is the maximum sequence length LingBot-Map can process without windowed mode?

Without windowed mode, the KV-cache is limited to approximately 320 frames due to the RoPE training range constraints. For longer outdoor videos, you must use --mode windowed to slide a fixed-size cache window across the sequence, resetting the cache at each window boundary.

How do I calculate the number of windows needed for my video?

Each window covers num_scale_frames + (window_size - num_scale_frames) * keyframe_interval frames. With the recommended defaults of 8 scale frames, window size 128, and interval 13, each window handles approximately 1,568 frames. Divide your total frame count by this value to estimate the number of windows, accounting for the overlap specified by --overlap_keyframes which ensures smooth transitions between windows.

Can I reuse sky masks for multiple rendering attempts?

Yes. Specify --sky_mask_dir to cache computed binary masks. The pipeline checks this directory via get_sky_artifact_dirs (lines 658–674) before recomputing, making subsequent renders instantaneous for the segmentation step. The masks are generated using the skyseg.onnx model and stored for reuse.

Why are my outdoor point clouds too sparse to see clearly?

Increase --point_size to between 0.2 and 0.5 (default is smaller). This parameter overrides cfg.render.point_size at line 1110 in batch_demo.py, making individual points larger in the final MP4 fly-through. For very sparse reconstructions, you may also want to adjust --vis_threshold and --conf_threshold to filter low-quality points.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}