# How BSSLoader Handles Image Loading and Resizing in lingbot-map

> Learn how BSSLoader in lingbot-map rescales RGB depth and mask images BSSLoader ensures consistent geometric transforms and updated camera intrinsics for all modalities.

- Repository: [Robbyant/lingbot-map](https://github.com/Robbyant/lingbot-map)
- Tags: how-to-guide
- Published: 2026-07-02

---

**BSSLoader automatically rescales RGB, depth, and mask images using a ResizeContext when loading data from BSS-formatted directories, ensuring all modalities share consistent geometric transforms and updated camera intrinsics.**

The `BSSLoader` class in the lingbot-map repository serves as the central data access layer for benchmark datasets stored in BSS format. When processing robotics perception data, developers need to understand how this loader handles image loading and resizing to maintain geometric accuracy across RGB frames, depth maps, and segmentation masks. The implementation couples file I/O with optional spatial transformations, allowing batch processing with consistent camera models.

## BSSLoader Architecture and Initialization

The loader acts as a bridge between raw BSS artifacts and processed tensors, orchestrating reads from RGB, depth, and mask directories while applying unified geometric transforms.

### Constructor and ResizeContext Storage

When instantiating `BSSLoader`, the class stores a `BSSArtifact` reference and an optional `ResizeContext` parameter. According to the source code in [`benchmark/benchmark/core/loader.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/benchmark/core/loader.py) (lines 30–46), if `resize_context` is `None` or its mode is set to `'none'`, the loader returns images at their native resolution without modification.

```python
loader = BSSLoader(artifact, resize_context=resize_ctx)  # resize_ctx may be None

```

This design allows the same loader instance to toggle between high-resolution analysis and downsampled benchmarking simply by changing the context parameter.

## Step-by-Step Image Loading Flow

The loading process follows a deterministic pipeline that queries metadata, iterates over frames, and conditionally applies resizing before returning lists of arrays.

### Frame Discovery via Metadata

Before loading pixels, `get_num_frames()` reads the [`.complete.json`](https://github.com/Robbyant/lingbot-map/blob/main/.complete.json) metadata file to determine the valid frame indices for iteration. This method, found at lines 56–74 in [`benchmark/benchmark/core/loader.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/benchmark/core/loader.py), ensures that the loader only attempts to read existing files and establishes the iteration range for subsequent list-loading methods.

### RGB Image Loading and Resizing

The `load_rgb_list()` method (lines 71–100) constructs file paths using zero-padded indexing (`f"{idx:06d}.png"`) and delegates actual file reading to `load_rgb()`, a thin wrapper around `cv2.imread`. When a `ResizeContext` is present and enabled, each loaded image is passed to `ResizeContext.apply()` before being appended to the result list.

This guarantees that all RGB frames in the returned list share identical dimensions defined by the resize parameters.

### Depth Map Loading with Nearest-Neighbor Interpolation

Depth data requires special handling to preserve metric precision. The `load_depth_list()` method (lines 106–130) loads `.exr` files via `load_exr()` and, when resizing is active, invokes `ResizeContext.apply_nearest()` instead of the standard `apply()` method.

**Nearest-neighbor interpolation** is critical here because linear or cubic interpolation would introduce invalid depth values at object boundaries, corrupting subsequent 3D back-projection calculations.

### Mask Loading and Boolean Conversion

For segmentation masks, `load_mask_list()` (lines 91–115) reads PNG files and converts them to boolean arrays. Like depth maps, masks use `ResizeContext.apply_nearest()` to avoid generating fractional opacity values that could break downstream boolean logic or contour detection algorithms.

## ResizeContext Internals and Transform Pipeline

The geometric transformation logic resides in [`benchmark/benchmark/geometry/resize.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/benchmark/geometry/resize.py), where `ResizeContext` encapsulates scaling factors, crop offsets, and canvas dimensions.

### Lazy Transform Computation

`ResizeContext.apply()` lazily initializes a `ResizeTransform` object via `_compute_resize_transform()` the first time an image of a specific size is encountered. This cached transform stores the mathematical operations required to map the original image dimensions to the target size, accounting for aspect ratio preservation and alignment constraints (e.g., `align=4` for tensor-friendly dimensions).

### The Three-Step Resize Pipeline

As implemented in lines 64–78 of [`resize.py`](https://github.com/Robbyant/lingbot-map/blob/main/resize.py), `_apply_resize_transform()` executes a rigid three-step sequence:

1. **Resize** – Scale the image using the computed factors
2. **Crop** – Remove pixels if the scaled image exceeds the target canvas
3. **Place** – Position the result on a zero-filled canvas of the exact target size

This pipeline ensures consistent results regardless of aspect ratio differences between the source data and target resolution.

## Intrinsics Adjustment for Geometric Consistency

When camera calibration data is loaded via `load_intrinsics()` (lines 145–158 in [`loader.py`](https://github.com/Robbyant/lingbot-map/blob/main/loader.py)), the loader automatically invokes `adjust_intrinsics()` using the same `ResizeTransform` applied to the images. This updates the focal lengths and principal point coordinates to match the resized image coordinates, preserving the mathematical relationship between pixel space and camera space.

Without this adjustment, 3D reconstruction algorithms would use incorrect projection matrices, leading to skewed point clouds and inaccurate pose estimation.

## Practical Code Examples

The following patterns demonstrate how to configure `BSSLoader` for different processing scenarios:

```python
from benchmark.core.storage import BSSArtifact
from benchmark.core.loader import BSSLoader
from benchmark.geometry.resize import ResizeContext

# Load RGB images at native resolution (no resizing)

artifact = BSSArtifact("/path/to/bss_dataset")
loader = BSSLoader(artifact)                     # resize_context is None → no scaling

rgb_images = loader.load_rgb_list()            # List[ndarray] of shape (H, W, 3)

# Load images scaled to fit an area budget (e.g., max 320×240 pixels)

resize_ctx = ResizeContext(mode="area_budget", area_budget=76800, align=4)
loader = BSSLoader(artifact, resize_context=resize_ctx)
rgb_images = loader.load_rgb_list()            # Resized RGB

depth_maps = loader.load_depth_list()          # Resized depth (nearest-neighbor)

# Load adjusted intrinsics for the same resize transform

intrinsics = loader.load_intrinsics()            # Already corrected for the resize

```

## Summary

- **BSSLoader** couples file I/O from [`benchmark/benchmark/core/loader.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/benchmark/core/loader.py) with the geometric transforms defined in [`benchmark/benchmark/geometry/resize.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/benchmark/geometry/resize.py) to produce consistently sized output.
- **ResizeContext** applies to RGB via interpolation-based scaling, but uses **nearest-neighbor** for depth and masks to preserve data integrity.
- **Camera intrinsics** are automatically adjusted using the same `ResizeTransform`, ensuring that 3D geometry remains valid after downsampling.
- The loader determines frame counts by parsing [`.complete.json`](https://github.com/Robbyant/lingbot-map/blob/main/.complete.json) metadata, preventing attempts to load non-existent files.
- When `resize_context` is `None` or mode is `'none'`, images return at their native resolution with no computational overhead.

## Frequently Asked Questions

### What image formats does BSSLoader support?

BSSLoader supports PNG files for RGB images and masks (loaded via `cv2.imread`), and OpenEXR (`.exr`) files for depth data (loaded via `load_exr()` in [`benchmark/io/image.py`](https://github.com/Robbyant/lingbot-map/blob/main/benchmark/io/image.py)). These formats preserve the necessary bit depth for computer vision tasks, with EXR specifically handling the floating-point precision required for metric depth.

### Why does BSSLoader use nearest-neighbor interpolation for depth maps?

Nearest-neighbor interpolation prevents the introduction of interpolated depth values at object boundaries. Linear or cubic interpolation would create fractional depth values that do not correspond to real surfaces in the scene, corrupting 3D back-projection calculations and point cloud generation.

### How does BSSLoader maintain geometric consistency when resizing images?

The loader maintains consistency by caching a `ResizeTransform` object that stores scaling factors and crop offsets. This same transform is applied to all modalities (RGB, depth, mask) and is also used to adjust camera intrinsics via `adjust_intrinsics()`, ensuring that the mathematical relationship between pixels and 3D space remains valid after resizing.

### What happens if ResizeContext is set to 'none' mode?

When `ResizeContext` is initialized with `mode='none'` or passed as `None` to the `BSSLoader` constructor, the loader skips all transformation logic. Images return at their native resolution read directly from disk, and intrinsics remain unmodified, providing zero-overhead access to the original dataset.