# How to Use SAM for Automatic Mask Generation: A Complete Guide

> Learn how to use SAM for automatic mask generation with facebookresearch segment anything. Generate segmentation masks for entire images efficiently without manual prompts. Get your complete guide now.

- Repository: [Meta Research/segment-anything](https://github.com/facebookresearch/segment-anything)
- Tags: how-to-guide
- Published: 2026-03-07

---

**Use the `SamAutomaticMaskGenerator` class from the `facebookresearch/segment-anything` repository to generate segmentation masks for an entire image without manual prompts by sampling a grid of points and running batch inference with quality filtering.**

The **Segment Anything Model (SAM)** provides a powerful pipeline for automatic mask generation that processes entire images without requiring user input prompts. This functionality is implemented in the `SamAutomaticMaskGenerator` class, which orchestrates point-grid sampling, multi-scale cropping, and intelligent filtering to produce high-quality object masks. Below is a comprehensive guide on how to use SAM for automatic mask generation, covering the underlying mechanics, implementation details, and configuration options.

## How SamAutomaticMaskGenerator Works

The automatic mask generation pipeline operates through a sophisticated multi-step process defined in [`segment_anything/automatic_mask_generator.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/automatic_mask_generator.py). Understanding these stages helps optimize the generator for your specific use case.

### Model Setup and Image Embedding

First, the SAM model is instantiated via `sam_model_registry` and wrapped in a `SamPredictor` for efficient embedding reuse. When processing begins, `SamPredictor.set_image` (defined in [`segment_anything/predictor.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/predictor.py)) computes the image embedding once, allowing rapid mask prediction for multiple point prompts without recomputing backbone features.

### Point Grid and Multi-Crop Strategy

The generator creates a grid of point prompts covering the entire image using `build_all_layer_point_grids`. When `crop_n_layers` is greater than 0, the image is split into overlapping crops via `generate_crop_boxes`, enabling detection of small objects that might be missed at full resolution. Each crop is processed independently with its own point grid scaled appropriately.

### Mask Prediction and Quality Filtering

For each crop, points are processed in batches (controlled by `points_per_batch`). The `SamPredictor.predict_torch` method generates masks, IoU predictions, and low-resolution logits. Masks undergo rigorous quality filtering based on:

- **Predicted IoU** (`pred_iou_thresh`): Filters masks by model confidence
- **Stability score** (`stability_score_thresh`): Measures mask quality under threshold perturbations
- **Box NMS** (`box_nms_thresh`): Removes duplicate masks within each crop

### Non-Maximum Suppression and Post-Processing

After processing all crops, a second NMS pass (`crop_nms_thresh`) resolves overlaps between crops, preferring masks from smaller crops to preserve fine details. If `min_mask_region_area` is specified, OpenCV removes small isolated regions and holes, followed by a final NMS pass to clean the results.

## Basic Usage: Python Implementation

To use SAM for automatic mask generation in Python, instantiate the generator with a loaded model and call the `generate` method:

```python
from segment_anything import SamAutomaticMaskGenerator, sam_model_registry
import cv2
import numpy as np

# ① Load the SAM model (ViT-H variant)

sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h_4b8939.pth")

# ② Create the automatic mask generator

mask_generator = SamAutomaticMaskGenerator(sam)

# ③ Load your image (HxWx3 uint8 array)

image = cv2.imread("input_image.jpg")
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# ④ Generate masks automatically

masks = mask_generator.generate(image)

# ⑤ Inspect results

print(f"Generated {len(masks)} masks")
print(masks[0].keys())  # ['segmentation', 'bbox', 'area', 'predicted_iou', ...]

```

The `generate` method returns a list of dictionaries, where each entry contains the binary segmentation mask, bounding box coordinates (XYWH format), mask area, predicted IoU, stability score, and the point prompt that generated it.

## Command-Line Interface for Batch Processing

For processing images without writing Python code, use the provided [`scripts/amg.py`](https://github.com/facebookresearch/segment-anything/blob/main/scripts/amg.py) utility:

```bash
python scripts/amg.py \
    --checkpoint sam_vit_h_4b8939.pth \
    --model-type vit_h \
    --input path/to/images/ \
    --output annotations.json

```

This script wraps `SamAutomaticMaskGenerator` with default parameters and outputs a JSON file following the schema documented in the repository README. It supports both single images and directories for batch processing.

## Advanced Configuration for Higher Recall

To maximize mask coverage for dataset annotation or dense object detection, tune the generator parameters when instantiating:

```python
mask_generator = SamAutomaticMaskGenerator(
    model=sam,
    points_per_side=64,          # Denser grid for more masks

    crop_n_layers=2,             # Two levels of cropping for small objects

    crop_n_points_downscale_factor=2,  # Fewer points per crop layer

    pred_iou_thresh=0.7,         # Lower threshold to keep more masks

    stability_score_thresh=0.9,  # Slightly relaxed stability requirement

    box_nms_thresh=0.7,          # Aggressive duplicate removal per crop

    crop_nms_thresh=0.7,         # Aggressive duplicate removal across crops

    min_mask_region_area=100,    # Remove tiny artifacts

    output_mode="coco_rle",      # COCO-compatible run-length encoding

)

```

These settings trade inference speed for recall, generating masks for small or ambiguous objects that default parameters might miss. The `output_mode="coco_rle"` option returns masks in COCO RLE format, compatible with `pycocotools` for evaluation.

## Summary

- **SamAutomaticMaskGenerator** in [`segment_anything/automatic_mask_generator.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/automatic_mask_generator.py) provides fully automatic mask generation without user prompts.
- The pipeline samples point grids across the image and optional multi-scale crops, batch processes them through `SamPredictor`, and filters results via IoU thresholds, stability scores, and NMS.
- Basic usage requires only loading a model checkpoint, instantiating the generator, and calling `generate(image)`.
- The [`scripts/amg.py`](https://github.com/facebookresearch/segment-anything/blob/main/scripts/amg.py) CLI tool enables batch processing without Python scripting.
- Constructor parameters like `points_per_side`, `crop_n_layers`, and `pred_iou_thresh` allow tuning the speed-recall trade-off for specific applications.

## Frequently Asked Questions

### What is the difference between SamPredictor and SamAutomaticMaskGenerator?

**SamPredictor** (defined in [`segment_anything/predictor.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/predictor.py)) is designed for interactive segmentation where you provide specific prompts such as points, boxes, or masks. It computes image embeddings once and allows rapid iteration on different prompts. **SamAutomaticMaskGenerator** (in [`segment_anything/automatic_mask_generator.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/automatic_mask_generator.py)) is built for fully automatic segmentation—it generates its own point prompts in a grid pattern across the entire image, handles multi-scale cropping internally, and returns all valid masks without requiring user input.

### How do I choose the right SAM model variant (vit_h, vit_l, vit_b) for automatic mask generation?

The choice depends on your speed-accuracy requirements and available GPU memory. **ViT-H (vit_h)** is the largest model with highest accuracy but slowest inference and requires the most VRAM. **ViT-L (vit_l)** offers a middle ground with good performance on most datasets. **ViT-B (vit_b)** is the fastest and most lightweight, suitable for real-time applications or edge devices with limited memory. For automatic mask generation, ViT-H is recommended when you need maximum recall for dataset annotation, while ViT-B works well for rapid prototyping.

### Why am I getting too many overlapping masks or duplicate detections?

Overlapping masks are a natural result of SAM's prompt-based architecture—different point prompts may generate masks for the same object. The `SamAutomaticMaskGenerator` applies **Non-Maximum Suppression (NMS)** at multiple stages to mitigate this: `box_nms_thresh` filters duplicates within each crop, and `crop_nms_thresh` resolves overlaps between different crop layers. If you still see too many duplicates, increase these threshold values (e.g., from 0.7 to 0.9) to be more aggressive in removing overlapping boxes. Alternatively, reduce `points_per_side` to generate fewer initial prompts.

### Can I use automatic mask generation on CPU-only machines?

Yes, but inference will be significantly slower. When loading the model, move it to CPU with `sam.to(device="cpu")` before passing it to `SamAutomaticMaskGenerator`. For CPU inference, consider using the **ViT-B** model variant and reducing `points_per_side` (e.g., to 16 or 32) and `crop_n_layers` (e.g., to 0 or 1) to minimize computation. The batch processing in `predict_torch` will still utilize vectorized operations, but expect processing times of several minutes per high-resolution image compared to seconds on GPU.