How to Load SAM Checkpoints: A Complete Guide to Segment Anything Model Weights

Load SAM checkpoints using the sam_model_registry dictionary in segment_anything/build_sam.py, which maps model types like "vit_h", "vit_l", or "vit_b" to constructor functions that automatically build the architecture and load the PyTorch .pth weights.

The Segment Anything Model (SAM) from Meta's facebookresearch/segment-anything repository requires properly loaded checkpoint files to perform image segmentation. When you need to load SAM checkpoints for inference or fine-tuning, the repository provides a streamlined registry system that handles architecture construction and weight loading in a single function call.

Understanding SAM Checkpoint File Structure

SAM checkpoints are standard PyTorch *.pth files containing the complete model weights for the full SAM architecture. These files include parameters for three core components: the image encoder (Vision Transformer backbone), the prompt encoder (handles point, box, and text prompts), and the mask decoder (generates the final segmentation masks).

Using the SAM Model Registry to Load Checkpoints

The segment_anything/build_sam.py file defines a sam_model_registry dictionary that maps model type strings to constructor functions. This registry eliminates manual architecture configuration when you load SAM checkpoints.

Available Model Types

The registry supports three model variants based on Vision Transformer (ViT) backbones:

• "vit_h" (default): ViT-Huge backbone with 632M parameters — highest accuracy, slowest inference
• "vit_l": ViT-Large backbone with 308M parameters — balanced accuracy and speed
• "vit_b": ViT-Base backbone with 89M parameters — fastest inference, lowest memory usage

The Checkpoint Loading Process

When you call a registry constructor with a checkpoint path, the code executes three critical steps defined in build_sam.py:

1. Architecture Construction: The _build_sam helper instantiates the model with the correct backbone size and component dimensions
2. Evaluation Mode: The model automatically calls sam.eval() to set dropout and batch normalization layers for inference
3. Weight Loading: The constructor opens the checkpoint file, deserializes the state dictionary with torch.load, and injects weights via sam.load_state_dict(state_dict)

# segment_anything/build_sam.py – checkpoint loading implementation

with open(checkpoint, "rb") as f:
    state_dict = torch.load(f)
sam.load_state_dict(state_dict)

Loading SAM Checkpoints in Python

Basic Usage with SamPredictor

The most common pattern for loading SAM checkpoints uses the SamPredictor wrapper class from segment_anything/predictor.py. This approach handles image preprocessing and provides a simple API for point and box prompts.

from segment_anything import SamPredictor, sam_model_registry

# Load SAM checkpoint using the registry

sam = sam_model_registry["vit_h"](checkpoint="/path/to/sam_vit_h_4b8939.pth")

# Wrap with predictor for preprocessing and inference

predictor = SamPredictor(sam)

# Set image (NumPy array, HWC, uint8)

predictor.set_image(my_image)

# Define prompts (point coordinates and labels)

point_coords = [[150, 200]]  # X, Y in pixel space

point_labels = [1]           # 1 = foreground, 0 = background

# Generate masks

masks, scores, low_res_logits = predictor.predict(
    point_coords=point_coords,
    point_labels=point_labels,
)

Loading Checkpoints for Automatic Mask Generation

For generating masks across an entire image without manual prompts, use the SamAutomaticMaskGenerator via the command-line script scripts/amg.py.

python scripts/amg.py \
    --checkpoint /path/to/sam_vit_h_4b8939.pth \
    --model-type vit_h \
    --input /path/to/input_image.jpg \
    --output /path/to/output_dir/

Internally, this script calls the same registry pattern:

from segment_anything import sam_model_registry, SamAutomaticMaskGenerator

sam = sam_model_registry[args.model_type](checkpoint=args.checkpoint)
mask_generator = SamAutomaticMaskGenerator(sam)
masks = mask_generator.generate(image)

Exporting Loaded Checkpoints to ONNX

To convert a loaded SAM checkpoint to ONNX format for deployment, use scripts/export_onnx_model.py. This script loads the checkpoint via the registry before exporting the mask decoder subgraph.

python scripts/export_onnx_model.py \
    --checkpoint /path/to/sam_vit_h_4b8939.pth \
    --model-type vit_h \
    --output sam_vit_h.onnx

The export script loads the checkpoint using the standard registry pattern in segment_anything/build_sam.py, then traces the mask decoder for ONNX compatibility.

Summary

• SAM checkpoints are PyTorch .pth files containing weights for the image encoder, prompt encoder, and mask decoder
• Use sam_model_registry from segment_anything/build_sam.py to map model types (vit_h, vit_l, vit_b) to constructor functions
• The registry automatically builds the correct architecture, sets evaluation mode, and loads the checkpoint state dictionary
• For interactive segmentation, wrap loaded models with SamPredictor from segment_anything/predictor.py
• For automatic mask generation, use SamAutomaticMaskGenerator via scripts/amg.py
• For deployment, export loaded checkpoints to ONNX using scripts/export_onnx_model.py

Frequently Asked Questions

What file format are SAM checkpoints?

SAM checkpoints are standard PyTorch state dictionary files saved with the .pth extension. They contain serialized tensors for all model parameters including the Vision Transformer image encoder, prompt encoder, and transformer-based mask decoder.

Which model type should I choose when loading SAM checkpoints?

Choose "vit_h" (ViT-Huge) for maximum accuracy when GPU memory is abundant, "vit_b" (ViT-Base) for fastest inference on resource-constrained devices, or "vit_l" (ViT-Large) for a balance between accuracy and speed. All three use the same checkpoint loading mechanism via sam_model_registry.

Can I load SAM checkpoints on CPU-only machines?

Yes, SAM checkpoints load on CPU by default unless you explicitly move the model to GPU with .to(device="cuda"). The SamPredictor wrapper automatically handles device placement for image preprocessing, but inference will run on CPU if CUDA is unavailable, though significantly slower than GPU inference.

How do I verify that a SAM checkpoint loaded correctly?

After loading via sam_model_registry, verify that sam.eval() returns the model in evaluation mode. The loading code in segment_anything/build_sam.py raises a RuntimeError if the checkpoint file is missing or corrupted, and load_state_dict will warn about missing or unexpected keys if the checkpoint architecture mismatches the selected model type.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}