What Is Sky Masking in LingBot-Map? Purpose and Implementation Explained

Sky masking in LingBot-Map automatically filters out sky pixels from RGB-D sequences to prevent unstructured, low-confidence regions from polluting 3D reconstructions, resulting in cleaner geometry and more reliable confidence weighting.

LingBot-Map is an open-source SLAM pipeline that constructs dense 3D point clouds from RGB-D image sequences. When cameras capture upward-facing views, the sky appears as an unstructured, low-texture region that can generate spurious points during back-projection. Sky masking solves this by detecting and removing these pixels before they enter the point cloud generation pipeline.

Why Sky Masking Matters in 3D Reconstruction

Unfiltered sky regions create three specific problems in SLAM systems:

• Geometric noise: Sky pixels back-project into random 3D points that float above the actual scene structure.
• Confidence corruption: Sky regions typically exhibit low confidence scores, which can trigger false positives in confidence-based filtering algorithms.
• Rendering overhead: Excess points from large sky regions increase GPU memory usage and slow down real-time visualisation.

By zero-ing out confidence scores for sky pixels before unprojection, LingBot-Map ensures that only foreground structures—walls, vegetation, and objects—contribute to the final map.

How Sky Masking Works in LingBot-Map

The masking pipeline operates in three stages implemented in lingbot_map/vis/sky_segmentation.py:

Step 1: ONNX-Based Sky Segmentation

LingBot-Map uses a pre-trained ONNX model (skyseg.onnx) to predict per-pixel sky probabilities. The run_skyseg function normalises the input image, runs inference, and returns an 8-bit score map.

According to the source code, this logic lives in lingbot_map/vis/sky_segmentation.py at lines 46-72. The function handles tensor conversion and model warmup, making it suitable for processing entire sequences.

Step 2: Mask Generation and Caching

The load_or_create_sky_masks function (lines 70-88) manages mask persistence. It checks for cached masks in a parallel directory structure (<image_folder>_sky_masks) before running inference on new images. Generated masks are stored as grayscale images, with optional visualisations saved to a separate directory for debugging.

This caching strategy prevents redundant inference when re-processing the same dataset, significantly reducing startup time for large sequences.

Step 3: Applying the Mask to Confidence Maps

The apply_sky_segmentation function (lines 73-76) performs the actual filtering. It loads the cached mask, thresholds it to a binary sky/no-sky map, and multiplies the confidence tensor (conf with shape (S, H, W)) by this binary mask. The multiplication step (lines 23-25) zeroes out confidence values where sky was detected, effectively removing those pixels from the subsequent unprojection step performed by unproject_depth_map_to_point_map in lingbot_map/utils/geometry.py.

Integration with the Visualisation Pipeline

Sky masking integrates seamlessly with the real-time visualiser through the viser_wrapper function in lingbot_map/vis/viser_wrapper.py (lines 82-85). When the mask_sky parameter is set to True, the wrapper automatically calls apply_sky_segmentation before converting depth maps to point clouds.

This design allows users to toggle sky masking without modifying the underlying SLAM logic, making it ideal for comparing reconstruction quality with and without sky filtering.

Practical Code Examples

Enabling Sky Masking in the Visualiser

from lingbot_map.vis.viser_wrapper import viser_wrapper

# `preds` contains the model output (images, depth, confidence, etc.)

viser_wrapper(
    pred_dict=preds,
    port=8080,
    mask_sky=True,               # Enable sky masking

    image_folder="data/images",  # Path to original RGB images

)

Direct Confidence Map Filtering

import numpy as np
from lingbot_map.vis.sky_segmentation import apply_sky_segmentation

# `conf` has shape (S, H, W) - confidence per frame

masked_conf = apply_sky_segmentation(
    conf,
    image_folder="data/images",   # Where raw images are stored

    skyseg_model_path="skyseg.onnx"
)

# masked_conf now has zero confidence where sky was detected

Pre-generating Sky Masks

from lingbot_map.vis.sky_segmentation import load_or_create_sky_masks

sky_masks = load_or_create_sky_masks(
    image_folder="data/images",
    sky_mask_dir="cache/sky_masks",
    sky_mask_visualization_dir="cache/sky_viz"
)

# Returns array of shape (S, H, W) ready for application to any confidence map

For a complete standalone example, see benchmark/sky_seg/seg_sky.py, which demonstrates batch processing of sky segmentation on benchmark datasets.

Summary

• Sky masking removes sky pixels from RGB-D sequences to prevent geometric noise and confidence corruption in 3D reconstructions.
• The pipeline uses an ONNX model (skyseg.onnx) via run_skyseg to generate per-pixel sky scores, implemented in lingbot_map/vis/sky_segmentation.py.
• Mask caching in <image_folder>_sky_masks avoids redundant inference on subsequent runs.
• Confidence multiplication in apply_sky_segmentation zeroes out sky regions before unprojection, ensuring only valid scene points enter the point cloud.
• The visualiser (viser_wrapper) provides a simple mask_sky flag to toggle this preprocessing without code changes.

Frequently Asked Questions

What happens if sky masking is disabled in LingBot-Map?

Without sky masking, upward-facing camera views generate floating point clusters in the 3D reconstruction where the sky was back-projected. These points exhibit low confidence scores, which can trigger false positives in confidence-based filtering and degrade the visual quality of the map.

Which model does LingBot-Map use for sky segmentation?

LingBot-Map uses a pre-trained ONNX model named skyseg.onnx loaded by the run_skyseg function. This model outputs per-pixel probability scores that are thresholded to create binary sky masks.

Where does LingBot-Map cache generated sky masks?

Generated masks are cached as grayscale images in a directory named <image_folder>_sky_masks, created automatically by load_or_create_sky_masks in the same parent directory as the source images. This allows the system to skip re-inference on subsequent runs.

How does sky masking affect point cloud rendering performance?

Sky masking improves rendering performance by reducing the total point count sent to the GPU. When cameras spend significant time looking upward, removing sky pixels can substantially decrease memory usage and increase frame rates in the real-time visualiser.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}