# SAM Model Sizes Explained: vit_h, vit_l, and vit_b Comparison in Segment Anything

> Explore SAM model sizes vit_h, vit_l, and vit_b. Understand the trade-offs between accuracy and computational cost for the Segment Anything Model.

- Repository: [Meta Research/segment-anything](https://github.com/facebookresearch/segment-anything)
- Tags: deep-dive
- Published: 2026-03-07

---

**The Segment Anything Model (SAM) offers three Vision Transformer backbone sizes—vit_h (Huge), vit_l (Large), and vit_b (Base)—that trade off between segmentation accuracy and computational cost through varying embedding dimensions, transformer depths, and attention head counts.**

The facebookresearch/segment-anything repository implements these three SAM model sizes to support diverse deployment scenarios, from edge devices to high-end research workstations. Understanding the architectural differences between vit_h, vit_l, and vit_b enables you to select the optimal balance of inference speed and mask quality for your computer vision pipeline.

## SAM Model Size Specifications and Architecture Differences

The three variants differ primarily in embedding dimension, transformer depth, and attention head configuration, which directly impacts GPU memory consumption and inference latency.

| Model | Backbone | Embed Dim | Depth | # Heads | Global-Attention Indices |

|-------|----------|-----------|-------|--------|-------------------------|
| **vit_h** | ViT-Huge | 1280 | 32 | 16 | [7, 15, 23, 31] |
| **vit_l** | ViT-Large | 1024 | 24 | 16 | [5, 11, 17, 23] |
| **vit_b** | ViT-Base | 768 | 12 | 12 | [2, 5, 8, 11] |

The **global-attention indices** specify which transformer blocks use full global attention rather than windowed attention, affecting the receptive field and computational cost. **vit_h** provides the highest accuracy but requires the most memory, while **vit_b** offers the fastest inference with the smallest footprint.

## Where SAM Model Sizes Are Defined in the Source Code

The model configurations are hardcoded in [`segment_anything/build_sam.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/build_sam.py), where each size has a dedicated builder function that passes specific hyperparameters to the internal `_build_sam` helper.

The **vit_h** builder (lines 14-21) initializes the huge variant:

```python
def build_sam_vit_h(checkpoint=None):
    return _build_sam(
        encoder_embed_dim=1280,
        encoder_depth=32,
        encoder_num_heads=16,
        encoder_global_attn_indexes=[7, 15, 23, 31],
        checkpoint=checkpoint,
    )

```

The **vit_l** builder (lines 27-34) configures the large variant:

```python
def build_sam_vit_l(checkpoint=None):
    return _build_sam(
        encoder_embed_dim=1024,
        encoder_depth=24,
        encoder_num_heads=16,
        encoder_global_attn_indexes=[5, 11, 17, 23],
        checkpoint=checkpoint,
    )

```

The **vit_b** builder (lines 37-44) defines the base variant:

```python
def build_sam_vit_b(checkpoint=None):
    return _build_sam(
        encoder_embed_dim=768,
        encoder_depth=12,
        encoder_num_heads=12,
        encoder_global_attn_indexes=[2, 5, 8, 11],
        checkpoint=checkpoint,
    )

```

All three functions delegate to `_build_sam` (lines 55-107), which assembles the complete architecture including the image encoder, prompt encoder, and mask decoder.

The **model registry** (lines 47-52) maps string identifiers to these builders:

```python
sam_model_registry = {
    "default": build_sam_vit_h,
    "vit_h": build_sam_vit_h,
    "vit_l": build_sam_vit_l,
    "vit_b": build_sam_vit_b,
}

```

## How to Load and Use Different SAM Model Sizes

Selecting a model size involves choosing the appropriate key from `sam_model_registry` and loading the corresponding checkpoint file.

### Loading a Specific Model Size

```python
from segment_anything import sam_model_registry

# Select model size: "vit_h", "vit_l", or "vit_b"

model_type = "vit_l"
checkpoint_path = "sam_vit_l.pth"

# Load model

sam = sam_model_registry[model_type](checkpoint=checkpoint_path)
sam.eval()  # Set to evaluation mode

```

### Running Inference with SamPredictor

The `SamPredictor` class provides a high-level interface for generating masks:

```python
from segment_anything import SamPredictor
import cv2
import numpy as np

# Initialize predictor with loaded model

predictor = SamPredictor(sam)

# Load and preprocess image

image = cv2.imread("example.jpg")
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# Set image (computes embeddings once)

predictor.set_image(image)

# Define point prompt (x, y) and label (1=foreground)

point_coords = np.array([[500, 375]])
point_labels = np.array([1])

# Generate masks

masks, scores, logits = predictor.predict(
    point_coords=point_coords,
    point_labels=point_labels,
    multimask_output=True,
)

```

### Exporting to ONNX for Deployment

For production environments, convert any model size to ONNX format:

```bash
python -m segment_anything.scripts.export_onnx_model \
    --checkpoint sam_vit_b.pth \
    --model-type vit_b \
    --output sam_vit_b.onnx

```

## Summary

- The **facebookresearch/segment-anything** repository provides three SAM model sizes—**vit_h**, **vit_l**, and **vit_b**—differentiated by Vision Transformer backbone scale and configured in [`segment_anything/build_sam.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/build_sam.py).
- **vit_h** (Huge) offers the highest accuracy with 1280 embedding dimensions and 32 transformer blocks, while **vit_b** (Base) provides the fastest inference with 768 dimensions and 12 blocks.
- The `sam_model_registry` dictionary maps string keys to builder functions, allowing you to instantiate any size by changing the model type parameter.
- Select **vit_b** for edge deployment, **vit_l** for balanced performance, and **vit_h** for maximum accuracy on high-end GPUs.

## Frequently Asked Questions

### What is the difference between vit_h, vit_l, and vit_b in SAM?

The three variants differ in their Vision Transformer backbone capacity as defined in [`segment_anything/build_sam.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/build_sam.py). **vit_h** (Huge) uses 1280 embedding dimensions across 32 transformer blocks with 16 attention heads, providing the highest accuracy but requiring the most GPU memory. **vit_l** (Large) uses 1024 dimensions and 24 blocks, while **vit_b** (Base) uses 768 dimensions and 12 blocks, delivering the fastest inference with the smallest memory footprint.

### Which SAM model size should I use for my application?

Choose **vit_b** if you are deploying on edge devices or consumer GPUs with limited VRAM, as it requires approximately 4x less memory than vit_h. Use **vit_l** for production servers where you need strong accuracy without the extreme memory overhead of the huge variant. Reserve **vit_h** for research environments or high-end workstations where maximum segmentation quality is prioritized over inference speed.

### How do I switch between SAM model sizes in code?

You can switch models by changing the key passed to `sam_model_registry` in [`segment_anything/build_sam.py`](https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/build_sam.py). The registry accepts `"vit_h"`, `"vit_l"`, or `"vit_b"` as strings, mapping each to its respective builder function (`build_sam_vit_h`, `build_sam_vit_l`, or `build_sam_vit_b`). Simply load the corresponding checkpoint file and instantiate the model using the registry key that matches your hardware constraints.

### Can I export smaller SAM models like vit_b to ONNX for faster deployment?

Yes, the repository includes a dedicated export script at [`scripts/export_onnx_model.py`](https://github.com/facebookresearch/segment-anything/blob/main/scripts/export_onnx_model.py) that supports all three model sizes. When exporting **vit_b**, the resulting ONNX model will have significantly smaller file size and faster CPU inference compared to **vit_h**, making it ideal for deployment in resource-constrained environments. The export process remains identical across variants—simply specify `--model-type vit_b` when running the script.