How to Integrate Ultralytics YOLO Models with BoxMOT: A Complete Developer Guide

BoxMOT provides a unified detection and ReID pipeline that automatically detects and loads any Ultralytics YOLO model (e.g., yolov8n.pt, custom .pt files) through the DetectorReIDPipeline class, enabling seamless multi-object tracking without custom preprocessing code.

The mikel-brostrom/boxmot repository abstracts away the complexity of combining object detection with appearance-based ReID models. When you integrate Ultralytics YOLO models with BoxMOT, the framework handles model loading, inference orchestration, and tensor conversion automatically, letting you focus on tracking logic rather than boilerplate code.

How the Ultralytics YOLO Integration Works

BoxMOT implements a three-layer architecture to support Ultralytics models natively. The system detects Ultralytics weights automatically and routes inference through optimized wrappers.

Core Integration Components

The integration relies on three specific source files that form a detection backend abstraction:

• Ultralytics detector (boxmot/detectors/ultralytics.py) – A thin wrapper that loads models via the YOLO() constructor and implements the load → preprocess → process → postprocess flow required by the generic Detector base class.

• DetectorReIDPipeline (boxmot/engine/inference.py) – Also exported as YOLOInference, this high-level engine creates the YOLO object, registers inference callbacks, and optionally wraps ReID models for synchronized timing measurements.

• Detector base class (boxmot/detectors/detector.py) – Provides the generic contract including resolve_image, weight loading utilities, and the __call__ workflow that the Ultralytics detector inherits.

The Inference Pipeline Flow

When you instantiate a pipeline with an Ultralytics model path, the following sequence executes:

1. Dependency Resolution – Ultralytics is declared as an optional extra in pyproject.toml (ultralytics>=3.8.200). Install with uv sync --all-extras or pip install boxmot[ultralytics].

2. Automatic Model Detection – The is_ultralytics_model() helper (defined in boxmot/detectors/__init__.py) inspects the supplied path. When it returns True, DetectorReIDPipeline instantiates the model via YOLO(path_or_placeholder) at lines 38–39 of inference.py.

3. Inference Execution – Calling pipeline.predict(source) forwards to self.yolo.predict(...). The underlying Ultralytics YOLO object handles internal preprocessing, inference, and post-processing, returning standard Results objects.

4. Tensor Extraction – Utilities such as extract_detections() (lines 84–99 of inference.py) extract the bounding-box tensor from result.boxes.data and convert it to a NumPy array with shape (N, 6) containing [x1, y1, x2, y2, conf, cls]. The same conversion logic exists in Ultralytics.postprocess() (lines 44–53 of ultralytics.py).

5. Performance Monitoring – The pipeline optionally records Ultralytics timing information via result.speed into a TimingStats object (lines 34–41). When ReID models are supplied, they are wrapped in TimedReIDModel to capture feature-extraction latency.

Because the Ultralytics detector conforms to the generic Detector API, you can swap it seamlessly with other backends such as YOLOX or RT-DETR without modifying downstream tracking code.

Running Inference with Ultralytics YOLO in BoxMOT

The YOLOInference class (alias for DetectorReIDPipeline) provides multiple interfaces for running detection, from single images to batch evaluation.

Single Image Detection

Use this pattern for real-time streaming or single-frame analysis:

from boxmot.engine.inference import YOLOInference
import cv2

# Accepts built-in model names or custom .pt checkpoints

yolo_path = "yolov8n.pt"  # or "path/to/my_custom_yolo.pt"

# Initialize pipeline; device can be "cpu", "cuda:0", etc.

pipeline = YOLOInference(yolo_path, device="cpu")

# Warm-up executes a dummy inference to optimize GPU memory allocation

pipeline.warmup()

# Run detection on a BGR numpy array (OpenCV default)

image = cv2.imread("sample.jpg")
results = next(pipeline.predict(image, stream=False))

# Convert to standard NumPy detection format

detections = pipeline.extract_detections(results)  # Shape: (N, 6)

print("Detections (x1, y1, x2, y2, conf, cls):")
print(detections)

Relevant source: Model instantiation at inference.py lines 31–38; extraction helper at lines 84–99.

Batch Processing for Evaluation

For dataset evaluation or offline processing, use the batch interface to amortize overhead across multiple images:

from boxmot.engine.inference import YOLOInference
import cv2
import glob

# Load image batch into memory

image_paths = glob.glob("data/images/*.jpg")
images = [cv2.imread(p) for p in image_paths]

# Initialize pipeline

pipeline = YOLOInference("yolov8s.pt", device="cpu")
pipeline.warmup()

# Process entire batch with consistent confidence and NMS thresholds

batch_results = pipeline.predict_batch(
    images=images,
    conf=0.3,
    iou=0.6,
    stream=False,
)

# Extract detections for each image

all_detections = [pipeline.extract_detections(r) for r in batch_results]

Relevant source: Batch processing logic at inference.py lines 44–55; timing aggregation at lines 84–90.

Adding ReID Models to the Pipeline

To enable appearance-based tracking, supply ReID weights alongside the YOLO model:

from boxmot.engine.inference import YOLOInference
import cv2

# Configure pipeline with both detection and ReID models

pipeline = YOLOInference(
    yolo_model_path="yolov8m.pt",
    reid_model_paths="reid_weights/resnet50.pt",  # Accepts list of paths

    device="cuda:0",
)

image = cv2.imread("sample.jpg")
result = next(pipeline.predict(image, stream=False))

# Get detection boxes

detections = pipeline.extract_detections(result)

# Extract ReID features for each detection (N x feature_dim)

features = pipeline.get_reid_features(detections[:, :4], image)
print("ReID feature shape:", features.shape)

Relevant source: ReID initialization at inference.py lines 60–85; feature extraction wrapper at lines 50–73.

Low-Level Detector Access

For custom workflows that bypass the high-level pipeline, instantiate the Ultralytics detector directly:

from boxmot.detectors.ultralytics import Ultralytics
import cv2

# Manual configuration of inference parameters

detector = Ultralytics(
    path="yolov8n.pt",
    device="cpu",
    conf=0.25,
    iou=0.45,
    imgsz=640,
)

# Direct __call__ returns NumPy array (N, 6)

boxes = detector(cv2.imread("sample.jpg"))
print("Detected boxes:", boxes)

Relevant source: Detection flow implementation at ultralytics.py lines 8–57.

Key Source Files and Architecture

Understanding these files helps when debugging or extending the integration:

• boxmot/detectors/ultralytics.py – Concrete implementation of the Ultralytics detector wrapper, handling the YOLO class interface and tensor post-processing.

• boxmot/detectors/detector.py – Abstract base class defining the Detector contract, including image resolution helpers and weight loading utilities.

• boxmot/engine/inference.py – Central DetectorReIDPipeline that routes model instantiation, provides predict and predict_batch methods, and manages timing statistics.

• boxmot/detectors/__init__.py – Contains is_ultralytics_model(), the heuristic used to identify Ultralytics-compatible weight files.

• pyproject.toml – Declares the ultralytics extra dependency (ultralytics>=3.8.200) required for this integration.

• boxmot/utils/timing.py – Implements TimingStats for aggregating inference latency data from both detection and ReID stages.

Summary

• BoxMOT automatically detects Ultralytics YOLO models via is_ultralytics_model() and instantiates them using the native YOLO() constructor.
• No manual preprocessing is required because the Ultralytics wrapper delegates image normalization and tensor conversion to the underlying library.
• The DetectorReIDPipeline (aliased as YOLOInference) provides unified interfaces for single-image, batch, and ReID-enhanced inference.
• Detection outputs are standardized to NumPy arrays with shape (N, 6) containing bounding boxes, confidence scores, and class IDs.
• Architecture is backend-agnostic; swapping between Ultralytics YOLO, YOLOX, or RT-DETR requires only changing the model path.

Frequently Asked Questions

Does BoxMOT support custom-trained Ultralytics YOLO models?

Yes. Any valid Ultralytics .pt checkpoint—including those trained on custom datasets—works with BoxMOT. Pass the absolute or relative path to your weights file (e.g., "runs/detect/train/weights/best.pt") to YOLOInference. The is_ultralytics_model() function identifies the file format, and the pipeline loads it via YOLO(path) at lines 38–39 of inference.py.

What Ultralytics YOLO versions are compatible with BoxMOT?

BoxMOT requires ultralytics>=3.8.200 as specified in pyproject.toml. This version constraint ensures compatibility with the YOLO class API and the Results object structure used by the extract_detections() method. Newer YOLOv8, YOLOv9, and YOLOv11 models following the Ultralytics training format are supported.

How does BoxMOT handle image preprocessing for Ultralytics models?

Preprocessing is handled internally by the Ultralytics library itself. When you call pipeline.predict(image), the request forwards to self.yolo.predict(...), which executes resizing, normalization, and batching according to the model's imgsz parameter. The Ultralytics wrapper in boxmot/detectors/ultralytics.py only post-processes the output tensors in postprocess() (lines 44–53), converting result.boxes.data tensors to NumPy arrays.

Can I use BoxMOT with other detectors besides Ultralytics YOLO?

Yes. The Detector base class in boxmot/detectors/detector.py abstracts the detection backend. BoxMOT includes implementations for YOLOX and RT-DETR in addition to Ultralytics. Because all detectors implement the same __call__ interface returning (N, 6) arrays, you can substitute model paths in DetectorReIDPipeline without changing your tracking or evaluation scripts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}