# Sky Masking for Outdoor Scenes in LingBot-Map: ONNX-Based Segmentation Guide

> Learn how LingBot-Map performs sky masking for outdoor scenes with its ONNX-based segmentation model. Automatically detect and zero out sky pixels effortlessly.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-28

---

**Yes, LingBot-Map performs sky masking for outdoor scenes using an ONNX-based segmentation model implemented in [`sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/sky_segmentation.py) that automatically detects and zeros out sky pixels in confidence maps.**

LingBot-Map is an open-source 3D reconstruction framework designed to handle challenging outdoor environments. The repository includes a dedicated **sky masking for outdoor scenes** pipeline that filters sky regions from confidence volumes to improve mapping accuracy and visualization quality.

## Core Sky Segmentation Architecture

The sky masking system centers on [[`lingbot_map/vis/sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py), which implements a complete segmentation workflow using a pretrained model hosted on HuggingFace.

### Model Acquisition and Initialization

The `download_skyseg_model()` function (lines 30-38) manages automatic model retrieval, verifying local availability before downloading the ONNX file from remote storage. This ensures the segmentation pipeline functions without manual configuration.

### Mask Generation and Caching

The `load_or_create_sky_masks()` function (lines 10-24) serves as the primary entry point for mask generation. This utility either loads cached masks from disk or processes input images through `segment_sky()` and `segment_sky_from_array()` to produce per-frame binary arrays. The implementation caches results to avoid redundant computation on subsequent runs.

### Confidence Map Filtering

To apply masks to reconstruction data, the `apply_sky_segmentation()` function (lines 73-78) multiplies the input confidence volume—typically a NumPy array of shape `(S, H, W)`—by a binary sky mask thresholded at 0.1. This operation sets sky pixel confidences to zero while preserving non-sky regions for downstream processing.

## Integration with Visualization Pipelines

The sky masking functionality is tightly coupled with LingBot-Map's visualization stack to ensure consistent rendering.

In [[`lingbot_map/vis/viser_wrapper.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py) (lines 21-84), confidence maps pass through `apply_sky_segmentation()` prior to Viser interface rendering. Similarly, [[`lingbot_map/vis/point_cloud_viewer.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/point_cloud_viewer.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/point_cloud_viewer.py) (lines 33-175) invokes the same masking routine when generating confidence overlays for point cloud displays, preventing sky artifacts from contaminating 3D visualizations.

## Practical Implementation Examples

### Masking Confidence Volumes

To filter sky regions from reconstruction confidence data:

```python
import numpy as np
from lingbot_map.vis.sky_segmentation import apply_sky_segmentation

# conf is a (S, H, W) confidence volume from the reconstruction pipeline

conf_masked = apply_sky_segmentation(
    conf,
    image_folder="/path/to/outdoor/images",
    skyseg_model_path="skyseg.onnx",  # Auto-downloaded if missing

    sky_mask_dir="/tmp/sky_masks",
    sky_mask_visualization_dir="/tmp/sky_viz"
)

```

### Generating Standalone Sky Masks

For custom preprocessing workflows, generate masks independently:

```python
from lingbot_map.vis.sky_segmentation import load_or_create_sky_masks

# Returns array of shape (S, H, W) with 1=non-sky, 0=sky

sky_masks = load_or_create_sky_masks(
    image_folder="/path/to/outdoor/images",
    skyseg_model_path="skyseg.onnx",
    sky_mask_dir="/tmp/sky_masks"
)

```

## Key Source Files and Functions

Understanding the module structure enables effective customization:

- **[[`lingbot_map/vis/sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py)**: Contains `download_skyseg_model()`, `load_or_create_sky_masks()`, and `apply_sky_segmentation()` implementing the core ONNX-based segmentation logic.
- **[[`lingbot_map/vis/viser_wrapper.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py)**: Integrates sky masking into the Viser visualization pipeline (lines 21-84).
- **[[`lingbot_map/vis/point_cloud_viewer.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/point_cloud_viewer.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/point_cloud_viewer.py)**: Applies sky masking for point cloud confidence overlays (lines 33-175).
- **[[`lingbot_map/vis/utils.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/utils.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/utils.py)**: Provides image I/O utilities supporting the segmentation workflow.

## Summary

- LingBot-Map implements **sky masking for outdoor scenes** through an ONNX-based segmentation model with automatic HuggingFace model downloading.
- The [[`sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/sky_segmentation.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py) module provides `apply_sky_segmentation()` for filtering confidence volumes and `load_or_create_sky_masks()` for standalone mask generation.
- Binary masks use a 0.1 threshold to identify sky pixels (0=sky, 1=non-sky), effectively removing atmospheric regions from 3D reconstruction data.
- Visualization components in [[`viser_wrapper.py`](https://github.com/Robbyant/lingbot-map/blob/main/viser_wrapper.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py) and [[`point_cloud_viewer.py`](https://github.com/Robbyant/lingbot-map/blob/main/point_cloud_viewer.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/point_cloud_viewer.py) automatically apply these masks during rendering.
- The system caches generated masks to disk, eliminating redundant computation across multiple sessions.

## Frequently Asked Questions

### Does LingBot-Map require manual annotation for sky masking?

No, the system uses a pretrained ONNX model accessed via `download_skyseg_model()`. The model downloads automatically from HuggingFace on first use, requiring no manual labeling or training data from the user.

### What input formats does the sky segmentation pipeline accept?

The pipeline accepts folders containing RGB image sequences. The `load_or_create_sky_masks()` function processes these through `segment_sky()` or `segment_sky_from_array()`, returning NumPy arrays of shape `(S, H, W)` where S represents the frame count.

### Can I adjust the threshold for sky detection sensitivity?

The `apply_sky_segmentation()` function applies a default threshold of 0.1 for binarizing probability maps. While the high-level API uses this fixed threshold, users can access raw segmentation probabilities through lower-level functions in [[`sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/sky_segmentation.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py) to implement custom thresholding logic.

### Is sky masking applied automatically in visualization tools?

Yes, both [[`viser_wrapper.py`](https://github.com/Robbyant/lingbot-map/blob/main/viser_wrapper.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py) and [[`point_cloud_viewer.py`](https://github.com/Robbyant/lingbot-map/blob/main/point_cloud_viewer.py)](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/point_cloud_viewer.py) automatically invoke `apply_sky_segmentation()` when processing confidence maps. This ensures sky regions are filtered before rendering without requiring explicit user intervention in standard workflows.