# What Is Sky Masking in LingBot-Map? Purpose and Implementation Explained

> Learn how sky masking in LingBot-Map refines 3D reconstructions by removing sky pixels. Achieve cleaner geometry and better confidence weighting.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-06

---

**Sky masking in LingBot-Map automatically filters out sky pixels from RGB-D sequences to prevent unstructured, low-confidence regions from polluting 3D reconstructions, resulting in cleaner geometry and more reliable confidence weighting.**

LingBot-Map is an open-source SLAM pipeline that constructs dense 3D point clouds from RGB-D image sequences. When cameras capture upward-facing views, the sky appears as an unstructured, low-texture region that can generate spurious points during back-projection. Sky masking solves this by detecting and removing these pixels before they enter the point cloud generation pipeline.

## Why Sky Masking Matters in 3D Reconstruction

Unfiltered sky regions create three specific problems in SLAM systems:

- **Geometric noise**: Sky pixels back-project into random 3D points that float above the actual scene structure.
- **Confidence corruption**: Sky regions typically exhibit low confidence scores, which can trigger false positives in confidence-based filtering algorithms.
- **Rendering overhead**: Excess points from large sky regions increase GPU memory usage and slow down real-time visualisation.

By zero-ing out confidence scores for sky pixels before unprojection, LingBot-Map ensures that only foreground structures—walls, vegetation, and objects—contribute to the final map.

## How Sky Masking Works in LingBot-Map

The masking pipeline operates in three stages implemented in [`lingbot_map/vis/sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py):

### Step 1: ONNX-Based Sky Segmentation

LingBot-Map uses a pre-trained ONNX model (`skyseg.onnx`) to predict per-pixel sky probabilities. The `run_skyseg` function normalises the input image, runs inference, and returns an 8-bit score map.

According to the source code, this logic lives in [`lingbot_map/vis/sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py) at lines 46-72. The function handles tensor conversion and model warmup, making it suitable for processing entire sequences.

### Step 2: Mask Generation and Caching

The `load_or_create_sky_masks` function (lines 70-88) manages mask persistence. It checks for cached masks in a parallel directory structure (`<image_folder>_sky_masks`) before running inference on new images. Generated masks are stored as grayscale images, with optional visualisations saved to a separate directory for debugging.

This caching strategy prevents redundant inference when re-processing the same dataset, significantly reducing startup time for large sequences.

### Step 3: Applying the Mask to Confidence Maps

The `apply_sky_segmentation` function (lines 73-76) performs the actual filtering. It loads the cached mask, thresholds it to a binary sky/no-sky map, and multiplies the confidence tensor (`conf` with shape `(S, H, W)`) by this binary mask. The multiplication step (lines 23-25) zeroes out confidence values where sky was detected, effectively removing those pixels from the subsequent unprojection step performed by `unproject_depth_map_to_point_map` in [`lingbot_map/utils/geometry.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/utils/geometry.py).

## Integration with the Visualisation Pipeline

Sky masking integrates seamlessly with the real-time visualiser through the `viser_wrapper` function in [`lingbot_map/vis/viser_wrapper.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/viser_wrapper.py) (lines 82-85). When the `mask_sky` parameter is set to `True`, the wrapper automatically calls `apply_sky_segmentation` before converting depth maps to point clouds.

This design allows users to toggle sky masking without modifying the underlying SLAM logic, making it ideal for comparing reconstruction quality with and without sky filtering.

## Practical Code Examples

### Enabling Sky Masking in the Visualiser

```python
from lingbot_map.vis.viser_wrapper import viser_wrapper

# `preds` contains the model output (images, depth, confidence, etc.)

viser_wrapper(
    pred_dict=preds,
    port=8080,
    mask_sky=True,               # Enable sky masking

    image_folder="data/images",  # Path to original RGB images

)

```

### Direct Confidence Map Filtering

```python
import numpy as np
from lingbot_map.vis.sky_segmentation import apply_sky_segmentation

# `conf` has shape (S, H, W) - confidence per frame

masked_conf = apply_sky_segmentation(
    conf,
    image_folder="data/images",   # Where raw images are stored

    skyseg_model_path="skyseg.onnx"
)

# masked_conf now has zero confidence where sky was detected

```

### Pre-generating Sky Masks

```python
from lingbot_map.vis.sky_segmentation import load_or_create_sky_masks

sky_masks = load_or_create_sky_masks(
    image_folder="data/images",
    sky_mask_dir="cache/sky_masks",
    sky_mask_visualization_dir="cache/sky_viz"
)

# Returns array of shape (S, H, W) ready for application to any confidence map

```

For a complete standalone example, see [`benchmark/sky_seg/seg_sky.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/sky_seg/seg_sky.py), which demonstrates batch processing of sky segmentation on benchmark datasets.

## Summary

- **Sky masking** removes sky pixels from RGB-D sequences to prevent geometric noise and confidence corruption in 3D reconstructions.
- The pipeline uses an **ONNX model** (`skyseg.onnx`) via `run_skyseg` to generate per-pixel sky scores, implemented in [`lingbot_map/vis/sky_segmentation.py`](https://github.com/Robbyant/lingbot-map/blob/main/lingbot_map/vis/sky_segmentation.py).
- **Mask caching** in `<image_folder>_sky_masks` avoids redundant inference on subsequent runs.
- **Confidence multiplication** in `apply_sky_segmentation` zeroes out sky regions before unprojection, ensuring only valid scene points enter the point cloud.
- The **visualiser** (`viser_wrapper`) provides a simple `mask_sky` flag to toggle this preprocessing without code changes.

## Frequently Asked Questions

### What happens if sky masking is disabled in LingBot-Map?

Without sky masking, upward-facing camera views generate floating point clusters in the 3D reconstruction where the sky was back-projected. These points exhibit low confidence scores, which can trigger false positives in confidence-based filtering and degrade the visual quality of the map.

### Which model does LingBot-Map use for sky segmentation?

LingBot-Map uses a pre-trained ONNX model named `skyseg.onnx` loaded by the `run_skyseg` function. This model outputs per-pixel probability scores that are thresholded to create binary sky masks.

### Where does LingBot-Map cache generated sky masks?

Generated masks are cached as grayscale images in a directory named `<image_folder>_sky_masks`, created automatically by `load_or_create_sky_masks` in the same parent directory as the source images. This allows the system to skip re-inference on subsequent runs.

### How does sky masking affect point cloud rendering performance?

Sky masking improves rendering performance by reducing the total point count sent to the GPU. When cameras spend significant time looking upward, removing sky pixels can substantially decrease memory usage and increase frame rates in the real-time visualiser.