# How to Enable Sky Masking in LingBot-Map and Install ONNX Runtime

> Enable LingBot-Map sky masking by installing ONNX Runtime and running the demo with --mask_sky. Automatically download the sky segmentation model for outdoor reconstructions.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-03

---

**Enable sky masking in LingBot-Map by installing the `onnxruntime` package and running the demo with the `--mask_sky` flag alongside an `--image_folder` path, which automatically downloads the sky segmentation model and filters sky points from outdoor reconstructions.**

LingBot-Map (Robbyant/lingbot-map) is an open-source mapping framework that supports optional sky masking to improve visual quality in outdoor scenes. This feature uses a lightweight ONNX model to segment sky regions and exclude them from point cloud generation. To enable sky masking in LingBot-Map and install the ONNX runtime, you need to configure the segmentation pipeline and install the appropriate inference library for your hardware.

## How Sky Masking Works in LingBot-Map

The sky masking pipeline is implemented across three core modules and operates lazily—only loading the ONNX model when first requested.

### Entry Points and Command-Line Flags

The `--mask_sky` argument is defined in [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) at line 409:

```python
parser.add_argument("--mask_sky", action="store_true",
                    help="Apply sky segmentation to filter out sky points")

```

At line 592, this flag is passed to the visualization wrapper:

```python
viser_wrapper(pred_dict, port=8080, mask_sky=args.mask_sky,
              image_folder=args.image_folder)

```

The offline rendering pipeline in [`demo_render/batch_demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/batch_demo.py) supports the same flag, allowing batch processing with sky filtering.

### Visualization Wrapper Integration

In [`lingbot_map/vis/viser_wrapper.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py), the function signature accepts the Boolean parameter:

```python
viser_wrapper(..., mask_sky: bool = False, image_folder: Optional[str] = None)

```

When `mask_sky` is `True` and an `image_folder` is provided, the wrapper calls `apply_sky_segmentation` (lines 82-85) after converting depth maps to world coordinates. The resulting confidence map modifies the per-point confidence values, dimming or omitting points where sky is detected.

### ONNX Model Inference and Caching

The segmentation logic resides in [`lingbot_map/vis/sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py). Key implementation details include:

- **Lazy Loading**: The ONNX inference session initializes only when `apply_sky_segmentation` is first called
- **Automatic Download**: The model downloads from `https://huggingface.co/JianyuanWang/skyseg/resolve/main/skyseg.onnx` on first use
- **Caching**: Masks are cached in `<image_folder>_sky_masks/` with a version stamp (`_SKYSEG_CACHE_VERSION`) to avoid recomputation
- **Thresholding**: Sky pixels are identified using `_SKYSEG_SOFT_THRESHOLD = 0.1`, where values below this threshold indicate sky regions

The function `segment_sky_from_array` processes each image through the ONNX runtime, rescales the output to the target resolution, and returns a confidence map that the visualizer uses to filter points.

## Installing ONNX Runtime for Sky Segmentation

The sky segmentation module imports `onnxruntime` (lines 19-22 in [`sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/sky_segmentation.py)). Without this dependency, the code prints a warning and disables masking.

Choose the appropriate package based on your hardware:

| Platform | Package | Install Command |
|----------|---------|-----------------|
| CPU Only | `onnxruntime` | `pip install onnxruntime` |
| NVIDIA GPU (CUDA) | `onnxruntime-gpu` | `pip install onnxruntime-gpu` |

The GPU version is recommended for large image sets or batch rendering pipelines, as it significantly accelerates inference speed.

## Step-by-Step Activation Guide

### 1. Install the ONNX Runtime

For CPU-only systems:

```bash
pip install onnxruntime

```

For CUDA-enabled GPUs:

```bash
pip install onnxruntime-gpu

```

### 2. Run the Interactive Demo with Sky Masking

Execute [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py) with the required flags:

```bash
python demo.py \
    --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/oxford \
    --mask_sky

```

The first run will download the `skyseg.onnx` model (approximately cached in `example/oxford_sky_masks/`), compute segmentation masks, and render the point cloud with sky points filtered.

### 3. Run Batch Rendering with Custom Cache Directories

For offline processing via [`demo_render/batch_demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo_render/batch_demo.py):

```bash
python demo_render/batch_demo.py \
    --video_path /data/drive.mp4 \
    --output_folder /data/outputs/drive/ \
    --model_path /path/to/lingbot-map-long.pt \
    --config demo_render/config/outdoor_drive.yaml \
    --mask_sky \
    --sky_mask_dir /data/outputs/drive/sky_masks \
    --sky_mask_visualization_dir /data/outputs/drive/sky_viz

```

### 4. Verify the Implementation

Check that the cache directory was created at `<image_folder>_sky_masks/` containing the computed masks. Subsequent runs will reuse these cached files unless the `_SKYSEG_CACHE_VERSION` changes in the source code.

## Summary

- **Sky masking** in LingBot-Map filters sky points from outdoor reconstructions using an ONNX segmentation model
- **Activation** requires both the `--mask_sky` flag and an `--image_folder` path to provide source images
- **Installation** of `onnxruntime` (CPU) or `onnxruntime-gpu` (CUDA) is mandatory for the feature to function
- **Caching** occurs automatically in `<image_folder>_sky_masks/`, avoiding redundant model inference on subsequent runs
- **Key files** involved are [`demo.py`](https://github.com/Robbyant/lingbot-map/blob/main/demo.py), [`lingbot_map/vis/viser_wrapper.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py), and [`lingbot_map/vis/sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py)

## Frequently Asked Questions

### What is the difference between `onnxruntime` and `onnxruntime-gpu`?

The `onnxruntime` package runs inference on CPU only, while `onnxruntime-gpu` leverages NVIDIA CUDA for hardware acceleration. For LingBot-Map, the GPU version processes large outdoor datasets significantly faster, though both produce identical segmentation results.

### Where does LingBot-Map store the downloaded sky segmentation model?

The model downloads automatically from the Hugging Face repository `JianyuanWang/skyseg` and caches in a directory named `<image_folder>_sky_masks/` adjacent to your image folder. You can specify a custom location using the `--sky_mask_dir` flag.

### Can I enable sky masking without providing an image folder?

No. The `viser_wrapper` function in [`lingbot_map/vis/viser_wrapper.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py) explicitly checks for both `mask_sky=True` and a valid `image_folder` path before calling `apply_sky_segmentation`. Without source images, the system cannot generate segmentation masks.

### How do I force recomputation of sky masks if the model updates?

Delete the cache directory (default: `<image_folder>_sky_masks/`) or increment the `_SKYSEG_CACHE_VERSION` constant in [`lingbot_map/vis/sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py). The code checks this version stamp on startup and regenerates masks if the cached version differs from the current code.