# What Is the SA-1B Dataset and How Does It Train SAM?

> Discover the SA-1B dataset, a massive collection of 1.1 billion segmentation masks on 11 million images, essential for training the Segment Anything Model (SAM). Learn how it enables promptable segmentation.

- Repository: [Meta Research/segment-anything](https://github.com/facebookresearch/segment-anything)
- Tags: deep-dive
- Published: 2026-03-07

---

**The SA-1B dataset comprises 1.1 billion segmentation masks across 11 million images, providing the visual diversity required to train the Segment Anything Model (SAM) for zero-shot, promptable segmentation.**

The SA-1B dataset serves as the massive training corpus behind Meta's Segment Anything Model (SAM). Released through the `facebookresearch/segment-anything` repository, this dataset provides the diverse visual supervision necessary for SAM's ability to segment virtually any object from prompts such as points, boxes, or text without task-specific fine-tuning.

## What Is the SA-1B Dataset?

The **SA-1B dataset** (Segment Anything 1 Billion) is a large-scale image collection specifically constructed to train foundation models for segmentation. Unlike traditional datasets with manual annotations, SA-1B was created using an automated pipeline that generated high-quality masks at unprecedented scale.

### Scale and Composition

SA-1B contains approximately **11 million images** paired with roughly **1.1 billion masks**. This massive scale provides the visual diversity required for **zero-shot generalization**, allowing SAM to recognize and segment objects it has never explicitly seen during training. Each image in the dataset is accompanied by multiple object masks representing various visual concepts, from common objects to obscure patterns.

### Data Format and Licensing

The mask data are stored as per-image JSON files using **COCO Run-Length Encoding (RLE)** format. According to the repository's [`README.md`](https://github.com/facebookresearch/segment-anything/blob/main/README.md), the dataset is covered by the **SA-1B Dataset Research License** and must be downloaded separately from Meta's dataset portal. The RLE format efficiently compresses binary masks into compact string representations suitable for large-scale storage.

## How SAM Trains on the SA-1B Dataset

SAM was trained directly on the masks contained within SA-1B, using the dataset's diversity to learn generalized segmentation features rather than class-specific patterns.

### Direct Supervision from Mask Annotations

The model was trained on the full corpus of 11 million images and 1.1 billion masks, treating each mask as a supervision signal for the **image encoder**, **prompt encoder**, and **mask decoder** architecture defined in [`segment_anything/modeling/sam.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/sam.py). This direct training approach enables SAM to predict valid masks for any prompt location, even in ambiguous scenarios where multiple objects overlap.

### The Automatic Mask Generation Pipeline

The same infrastructure used to create SA-1B is available in the repository via [`segment_anything/automatic_mask_generator.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/automatic_mask_generator.py). This module implements the grid-based point sampling strategy that originally generated the 1.1 billion masks. By running this generator across millions of images at multiple scales, the authors produced the diverse training set without manual annotation.

## Loading and Using SA-1B Data

Working with SA-1B requires decoding the COCO RLE format using `pycocotools`, then optionally running inference with SAM checkpoints pre-trained on this data.

### Decoding COCO RLE Masks

Each annotation file contains masks encoded in COCO RLE format. Use the following pattern to load and decode these masks into binary numpy arrays:

```python
import json
from pycocotools import mask as mask_utils

# Path to a per-image JSON file from SA-1B

json_path = "path/to/sa_image_00001.json"

with open(json_path) as f:
    data = json.load(f)

# Each entry in `annotations` holds a mask in COCO RLE format

annotation = data["annotations"][0]

# Decode the RLE to a binary mask (H × W numpy array)

binary_mask = mask_utils.decode(annotation["segmentation"])

```

### Running Inference with SAM

Once trained on SA-1B, SAM can generate masks for new images using prompts. The `SamPredictor` class in [`segment_anything/predictor.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/predictor.py) provides the inference interface:

```python
from segment_anything import SamPredictor, sam_model_registry

# Load a pre-trained checkpoint (trained on SA-1B)

sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h_4b8939.pth")
predictor = SamPredictor(sam)

# Set the image (NumPy H×W×3 array)

predictor.set_image(my_image)

# Single-point prompt example

point = [[250, 300]]          # (x, y) coordinates

label = [1]                   # 1 → foreground point

masks, scores, logits = predictor.predict(
    point_coords=point,
    point_labels=label,
    multimask_output=False,
)

# masks[0] contains the binary segmentation mask

```

### Regenerating Masks Automatically

To generate comprehensive masks for an entire image (replicating the SA-1B creation process), use `SamAutomaticMaskGenerator`:

```python
from segment_anything import SamAutomaticMaskGenerator, sam_model_registry

sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h_4b8939.pth")
mask_generator = SamAutomaticMaskGenerator(sam)

# Generates hundreds of masks across the image

all_masks = mask_generator.generate(my_image)

```

## Key Implementation Files in the Repository

The `facebookresearch/segment-anything` repository contains several critical files that implement the training infrastructure and data handling for SA-1B:

- **[`segment_anything/predictor.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/predictor.py)** – Implements `SamPredictor`, the core inference class that runs the image encoder and mask decoder trained on SA-1B.

- **[`segment_anything/automatic_mask_generator.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/automatic_mask_generator.py)** – Contains `SamAutomaticMaskGenerator`, the pipeline used to generate the 1.1 billion masks in SA-1B through grid-based point prompting.

- **[`segment_anything/modeling/sam.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/sam.py)** – Defines the SAM architecture including the Vision Transformer image encoder and the lightweight mask decoder.

- **[`README.md`](https://github.com/facebookresearch/segment-anything/blob/main/README.md)** (Dataset section) – Documents the SA-1B download location, licensing requirements, and data format specifications.

## Summary

- The **SA-1B dataset** contains 1.1 billion masks from 11 million images, making it the largest segmentation dataset available for research.
- Masks are stored in **COCO RLE format** within per-image JSON files and require `pycocotools` for decoding.
- SAM was **trained directly on SA-1B masks**, enabling its zero-shot segmentation capabilities across diverse visual domains.
- The **[`automatic_mask_generator.py`](https://github.com/facebookresearch/segment-anything/blob/main/automatic_mask_generator.py)** module implements the same pipeline used to create the dataset.
- The dataset is subject to the **SA-1B Dataset Research License** and hosted externally from the code repository.

## Frequently Asked Questions

### How large is the SA-1B dataset?

The SA-1B dataset contains approximately 11 million images with 1.1 billion associated segmentation masks. This makes it one of the largest publicly available image segmentation datasets by mask count, providing the scale necessary for training foundation models like SAM.

### What format are the masks stored in?

Masks are encoded using **COCO Run-Length Encoding (RLE)** format, stored as strings within per-image JSON files. To work with these masks, you must use `pycocotools.mask.decode()` to convert the RLE strings into binary numpy arrays representing the segmentation bitmaps.

### Can I use SA-1B for commercial applications?

No, the SA-1B dataset is distributed under the **SA-1B Dataset Research License**, which restricts usage to non-commercial research purposes. You must agree to these terms when downloading the dataset from Meta's portal, separate from the Apache-2.0 licensed code in the repository.

### How does SA-1B enable zero-shot segmentation?

By training on 1.1 billion diverse masks covering a vast range of visual concepts, SAM learns generalizable features for "what constitutes an object" rather than memorizing specific classes. This allows the model, as implemented in [`segment_anything/predictor.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/predictor.py), to segment novel objects during inference based solely on geometric prompts like points or bounding boxes.