# How to Use the Offline Batch Rendering Pipeline in LingBot-Map

> Master the offline batch rendering pipeline in LingBot-Map. Process video sequences and image folders efficiently through four key stages for stunning output.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-03

---

**The offline batch rendering pipeline in LingBot-Map processes long video sequences and large image folders through a four-stage workflow orchestrated by [`demo_render/batch_demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/batch_demo.py): data ingestion, model inference via streaming or windowed modes, prediction export as `.npz` files, and headless RGBD rendering into MP4 videos or GLB assets.**

LingBot-Map (available at Robbyant/lingbot-map) provides a dedicated offline pipeline for processing sequences that exceed the memory constraints of the interactive [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) viewer. By running [`demo_render/batch_demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/batch_demo.py) from the command line, you can extract frames from video sources, run inference on canonical-sized crops, and render high-quality fly-through videos without requiring a display attached to the machine.

## Understanding the Four-Stage Pipeline

The offline batch rendering pipeline operates through four distinct stages that transform raw media into final rendered outputs.

### Stage 1: Data Ingestion and Preprocessing

The pipeline accepts input via either `--video_path` for video files or `--input_folder` for directories of images. When processing video, the `extract_frames_from_video` function uses OpenCV to extract frames on-the-fly, optionally caching them as PNGs to `--save_frames_dir` for reuse across runs. For image folders, `list_image_paths` discovers files and filters them by range, stride, or explicit `--first_k` and `--last_k` arguments. All frames are then processed by `load_and_preprocess_images` in [`lingbot_map/utils/load_fn.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/utils/load_fn.py), which crops inputs to a canonical size and pads them to multiples of the model's patch size.

### Stage 2: Streaming vs. Windowed Inference

The `run_inference` function in [`demo_render/batch_demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/batch_demo.py) supports two execution modes controlled by the `--mode` argument.

**Streaming inference** (`--mode streaming`) calls `model.inference_streaming` and maintains a KV-cache for continuous frame sequences. Control memory usage by adjusting `--keyframe_interval`, which trims the cache every N frames.

**Windowed inference** (`--mode windowed`) calls `model.inference_windowed` and splits the sequence into overlapping windows. Configure the effective overlap using `--overlap_keyframes` or specify absolute pixel dimensions with `--overlap_size`. Set the window length with `--window_size`.

### Stage 3: Prediction Export and Caching

After inference, per-frame predictions are serialized via `save_predictions_npz` as a directory of `.npz` files. This layout enables fast parallel I/O when the rendering stage calls `load_predictions_from_npz`. Enable this export by including `--save_predictions` in your command. If you have previously saved predictions, you can bypass inference entirely by passing `--load_predictions` followed by the path to the `.npz` file.

### Stage 4: Headless RGBD Rendering

The final stage invokes `render_with_pipeline` to execute the **rgbd-render** pipeline. A `PipelineConfig` object is first built from an optional YAML preset (`--config`) and then overridden by any CLI flags. The pipeline performs CUDA-accelerated voxelization via `voxelize_frame` (from `demo_render/render_cuda_ext`), constructs a virtual camera path using `rgbd_render.camera.build_camera_path`, and adds optional overlays including trajectory trails, frustum visualization, and sky-mask overlays. The `OfflinePipeline` class writes the final output as an MP4 video. Enable sky masking with `--mask_sky`, which utilizes the ONNX segmentation model in [`lingbot_map/vis/sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py).

## Key Implementation Files

Understanding the source structure helps with debugging and extending the pipeline:

- **[`demo_render/batch_demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/batch_demo.py)**: Main entry point handling argument parsing, scene discovery, and stage orchestration.
- **[`lingbot_map/utils/load_fn.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/utils/load_fn.py)**: Contains `load_and_preprocess_images` for canonical cropping and resizing.
- **[`lingbot_map/vis/sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py)**: ONNX-based sky segmentation for the `--mask_sky` option.
- **`demo_render/render_cuda_ext/`**: CUDA extensions for `voxelize_frame` and frustum culling.
- **[`rgbd_render/pipeline/builder.py`](https://github.com/Robbyant/lingbot-map/blob/main/rgbd_render/pipeline/builder.py)** and **[`rgbd_render/pipeline/offline.py`](https://github.com/Robbyant/lingbot-map/blob/main/rgbd_render/pipeline/offline.py)**: Build scene representations and manage headless MP4 encoding.
- **`demo_render/config/*.yaml`**: Presets like [`indoor.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/indoor.yaml) and [`outdoor_drive.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/outdoor_drive.yaml) defining default camera paths and rendering parameters.

## Running the Offline Pipeline

Below are three typical usage patterns. Replace placeholder paths with your actual directories.

### Batch Processing Multiple Scenes

Process a folder where each subdirectory contains a scene:

```bash
python demo_render/batch_demo.py \
    --input_folder /data/scenes \
    --output_folder /data/outputs \
    --model_path /path/to/lingbot-map-long.pt \
    --mode streaming \
    --keyframe_interval 2 \
    --mask_sky \
    --save_predictions

```

### Processing a Single Video

Extract frames on-the-fly from a video file using windowed inference:

```bash
python demo_render/batch_demo.py \
    --video_path /data/video/indoor_travel.MP4 \
    --output_folder /data/outputs \
    --model_path /path/to/lingbot-map.pt \
    --config demo_render/config/indoor.yaml \
    --mode windowed \
    --window_size 128 \
    --overlap_keyframes 8 \
    --keyframe_interval 13 \
    --mask_sky \
    --save_predictions \
    --save_glb

```

### Rendering from Cached Predictions

Skip inference and render directly from previously saved `.npz` files:

```bash
python demo_render/batch_demo.py \
    --load_predictions /data/outputs/indoor_travel.npz \
    --output_folder /data/outputs \
    --config demo_render/config/indoor.yaml

```

## Configuration and Optimization

### Managing Memory with Keyframe Intervals

The `--keyframe_interval` parameter controls how frequently the KV-cache is trimmed during streaming inference. Lower values reduce memory consumption but may decrease temporal consistency. For windowed mode, ensure `--window_size` fits within your GPU memory, and use `--overlap_keyframes` to maintain smooth transitions between windows.

### YAML Presets and Camera Paths

Configuration files in `demo_render/config/` define default camera trajectories, voxel sizes, and overlay settings. Pass a preset with `--config`, then override specific values via CLI flags. For example, [`indoor.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/indoor.yaml) typically configures tighter camera paths and smaller voxel sizes compared to [`outdoor_drive.yaml`](https://github.com/Robbyant/lingbot-map/blob/main/outdoor_drive.yaml).

## Summary

- The offline batch rendering pipeline in LingBot-Map is orchestrated by [`demo_render/batch_demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/batch_demo.py) and designed for headless processing of long sequences.
- **Data ingestion** supports both video files and image folders, with preprocessing handled by [`lingbot_map/utils/load_fn.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/utils/load_fn.py).
- **Inference** supports streaming (with KV-cache management) and windowed (with overlap control) modes.
- **Predictions** are cached as `.npz` files via `save_predictions_npz`, enabling fast re-rendering without recomputing inference.
- **Rendering** uses the `rgbd_render` pipeline with CUDA-accelerated voxelization (`voxelize_frame`) and outputs MP4 videos or GLB assets.

## Frequently Asked Questions

### What is the difference between streaming and windowed inference in LingBot-Map?

**Streaming inference** (`--mode streaming`) processes the entire sequence continuously while maintaining a KV-cache in memory, making it suitable for long videos with the `--keyframe_interval` parameter to limit cache growth. **Windowed inference** (`--mode windowed`) splits the input into discrete chunks of size `--window_size` with configurable overlap (`--overlap_keyframes`), which is more memory-efficient for very high-resolution inputs but requires careful tuning of overlap to prevent boundary artifacts.

### How can I process videos that are too large to fit in GPU memory?

Use **windowed inference** with a reduced `--window_size` and increase the `--overlap_keyframes` to maintain temporal consistency between chunks. Alternatively, use **streaming inference** with a low `--keyframe_interval` (e.g., 2-4) to aggressively trim the KV-cache. The preprocessing step in `load_and_preprocess_images` automatically handles canonical cropping to reduce per-frame memory footprint.

### Can I reuse depth predictions without rerunning the neural network?

Yes. Include `--save_predictions` during your initial run to export `.npz` files via `save_predictions_npz`. On subsequent runs, pass `--load_predictions /path/to/file.npz` to skip the inference stage entirely and proceed directly to rendering or GLB export. This is significantly faster when experimenting with different camera paths or rendering configurations.

### What hardware is required for the CUDA voxelization step?

The `voxelize_frame` operation in `demo_render/render_cuda_ext` requires an NVIDIA GPU with CUDA support. The pipeline falls back to CPU implementations if the CUDA extensions are not built, but this drastically reduces performance when rendering long sequences. Ensure your environment has the CUDA toolkit installed and the extensions compiled before running the rendering stage.