Pre-trained Models for LingBot-Map: DINOv2 Integration Guide

LingBot-Map supports DINOv2 pre-trained weights via the pretrained_path argument in model constructors, automatically falling back to random initialization if the checkpoint path is missing or invalid.

The LingBot-Map repository provides a lightweight vision mapping framework that leverages external DINOv2 checkpoints rather than bundling pre-trained weights. According to the source code in Robbyant/lingbot-map, users can initialize transformer blocks with official Meta AI DINOv2 weights by providing a local filesystem path to the model classes in lingbot_map/models/.

Available Pre-trained Models for LingBot-Map

LingBot-Map exclusively supports DINOv2 pre-trained weights as its pre-trained model option. The repository does not bundle any checkpoint files—this design keeps the installation lightweight while allowing integration with full-size vision transformer backbones.

Users must download DINOv2 checkpoints (such as dinov2_vits14_pretrain.pth) from the official Meta AI release and provide the local path at runtime. When a valid path is supplied, the model loads these weights to initialize the transformer blocks; otherwise, the constructor catches the exception and initializes with random weights after logging a warning.

How to Load DINOv2 Weights in LingBot-Map

Supported Model Classes

The pretrained_path parameter is exposed in the following model implementations:

• lingbot_map/models/gct_stream.py – The primary GCTStream model class
• lingbot_map/models/gct_stream_window.py – Windowed attention variant with identical weight loading support
• lingbot_map/models/gct_stream_window_v2.py – Updated stream-window model maintaining the same pre-trained weight handling

Checkpoint Loading Implementation

As implemented in lingbot_map/aggregator/base.py, the aggregator loads the checkpoint by calling torch.load(pretrained_path) and uses the state dict to initialize the transformer blocks. The loader logs successful weight initialization and catches file-not-found or corruption errors to trigger random initialization with a logged warning.

from pathlib import Path
from lingbot_map.models import gct_stream

# Path to a downloaded DINOv2 checkpoint (e.g., "dinov2_vits14_pretrain.pth")

pretrained_ckpt = Path("/my/models/dinov2_vits14_pretrain.pth")

# Initialize the model with pre-trained weights

model = gct_stream.GCTStream(
    d_model=256,
    depth=12,
    pretrained_path=str(pretrained_ckpt),
)

# Forward pass with batch of images [B, 3, H, W]

features = model(images)

If pretrained_path is omitted or points to a non-existent file, the model completes initialization using random weights.

Model Architecture and File Structure

The pre-trained weight integration spans several key files:

• lingbot_map/models/gct_stream.py – Forwards the pretrained_path argument to the underlying aggregator
• lingbot_map/aggregator/base.py – Contains the torch.load() logic and weight initialization sequence
• lingbot_map/models/gct_stream_window.py and gct_stream_window_v2.py – Variant architectures supporting the same checkpoint format
• README.md – Documents the pipeline and confirms DINOv2 compatibility

This architecture ensures that pre-trained models for LingBot-Map remain optional dependencies rather than repository bloat, while providing seamless integration for users who download the official DINOv2 releases.

Summary

• LingBot-Map exclusively supports DINOv2 pre-trained weights; no other checkpoints are bundled with the repository
• Provide the local checkpoint path via the pretrained_path argument in GCTStream and variant constructors
• The loading mechanism in lingbot_map/aggregator/base.py uses standard torch.load() with automatic fallback to random initialization
• Supported model files include gct_stream.py, gct_stream_window.py, and gct_stream_window_v2.py

Frequently Asked Questions

Does LingBot-Map include pre-trained weights in the repository?

No. The repository maintains a lightweight footprint by excluding large checkpoint files. Users must download DINOv2 weights separately from the official Meta AI release and provide the local path via the pretrained_path argument when constructing the model.

Which model classes support the pretrained_path argument?

The pretrained_path parameter is available in lingbot_map/models/gct_stream.py, lingbot_map/models/gct_stream_window.py, and lingbot_map/models/gct_stream_window_v2.py. All three classes forward the path to the aggregator base class for weight loading.

What happens if the DINOv2 checkpoint path is invalid?

The model constructor catches the exception when torch.load() fails in lingbot_map/aggregator/base.py, logs a warning message, and initializes the transformer blocks with random weights. This ensures the model remains functional, though performance will differ from the pre-trained state.

Can I use vision transformers other than DINOv2?

The current implementation in lingbot_map/aggregator/base.py is specifically designed for DINOv2 checkpoint structures and key mappings. While the architecture might theoretically support other vision transformers with compatible weight dictionaries, the repository documentation and implementation explicitly target DINOv2 compatibility only.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}