How to Integrate Custom Object Detectors with BoxMOT: A Complete Implementation Guide

To integrate custom object detectors with BoxMOT, subclass the Detector base class in boxmot/detectors/detector.py, implement the four required methods (_load_model, preprocess, process, postprocess), register a unique model-type marker in boxmot/detectors/__init__.py, and pass a model path containing that marker to DetectorReIDPipeline.

BoxMOT (mikel-brostrom/boxmot) provides a modular inference pipeline that supports Ultralytics YOLO, YOLOX, and RT-DETR out of the box. When you need to integrate custom object detectors with BoxMOT—whether TorchScript exports, ONNX models, or custom architectures—the framework exposes a clean extension point through the Detector abstract base class that maintains full compatibility with the ReID and tracking pipeline.

Understanding the Detector Architecture

The BoxMOT inference engine relies on a strategy pattern to auto-select detection backends based on model filenames. In boxmot/detectors/__init__.py, the get_yolo_inferer function dispatches to the appropriate detector class by checking filename markers. For custom integration, you must implement the same interface that built-in detectors use.

The base class in boxmot/detectors/detector.py defines the contract that all detectors must follow. It provides a generic __call__ implementation that orchestrates the inference workflow: resolving image paths, preprocessing inputs, running forward passes, and postprocessing raw outputs into the standard BoxMOT format.

Step 1: Create Your Custom Detector Class

Subclass Detector and implement the four abstract methods that handle the complete inference lifecycle.

Implement the Four Required Methods

Create a new file boxmot/detectors/mydetector.py with the following structure:

from pathlib import Path
import cv2
import numpy as np
import torch

from boxmot.detectors.detector import Detector


class MyDetector(Detector):
    """
    Custom detector implementing the BoxMOT Detector interface.
    """

    def _load_model(self, path: str):
        """Load model weights. Supports TorchScript, ONNX, or custom formats."""
        if not Path(path).exists():
            raise FileNotFoundError(f"Model not found: {path}")
        return torch.jit.load(path, map_location="cpu")

    def preprocess(self, image: np.ndarray, **kwargs):
        """Convert BGR numpy image to normalized tensor with batch dimension."""
        img = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
        return torch.from_numpy(img).permute(2, 0, 1).unsqueeze(0)

    def process(self, tensor, **kwargs):
        """Execute forward pass. Returns raw predictions."""
        with torch.no_grad():
            return self.model(tensor)

    def postprocess(self, raw, **kwargs):
        """
        Convert raw predictions to BoxMOT standard format.
        Returns (N, 6) array: [x1, y1, x2, y2, confidence, class_id].
        """
        if raw.shape[0] == 0:
            return np.empty((0, 6))

        # Adapt to your model's specific output format

        boxes = raw[:, :4]  # x_center, y_center, width, height

        scores = raw[:, 4:5]
        classes = raw[:, 5:6]

        # Convert xywh to xyxy

        x1 = boxes[:, 0] - boxes[:, 2] / 2
        y1 = boxes[:, 1] - boxes[:, 3] / 2
        x2 = boxes[:, 0] + boxes[:, 2] / 2
        y2 = boxes[:, 1] + boxes[:, 3] / 2

        return np.concatenate([x1, y1, x2, y2, scores, classes], axis=1)

The base Detector class provides the __call__ method that automatically sequences these steps: preprocess → process → postprocess. Your implementation must return a NumPy array of shape (N, 6) containing bounding boxes in [x1, y1, x2, y2, confidence, class_id] format.

Step 2: Register the Model Type Marker

To enable auto-selection when instantiating DetectorReIDPipeline, register a filename marker and detection strategy in boxmot/detectors/__init__.py.

Adding a Detection Strategy

Edit boxmot/detectors/__init__.py to add your marker and strategy entry:

# Define unique filename markers (substring match)

MYDET_MODELS = {"mydet"}

def is_mydet_model(yolo_name):
    """Check if model filename contains the custom marker."""
    return any(marker in str(yolo_name) for marker in MYDET_MODELS)


def get_yolo_inferer(yolo_model):
    """
    Returns a tuple of (extra_pip_deps, extra_install_args, module_path, class_name)
    for the appropriate detector based on model filename.
    """
    strategies = [
        # Existing built-in strategies...

        (is_ultralytics_model, (), {}, "boxmot.detectors.ultralytics", "Ultralytics"),
        (is_yolox_model, (), {}, "boxmot.detectors.yolox", "YoloX"),
        (is_rtdetr_model, (), {}, "boxmot.detectors.rtdetr", "RTDETR"),
        # Add your custom strategy

        (
            is_mydet_model,                     # Condition function

            (),                                 # Extra pip dependencies (empty if none)

            {},                                 # Extra install arguments

            "boxmot.detectors.mydetector",      # Module import path

            "MyDetector",                       # Class name to instantiate

        ),
    ]
    
    for condition, deps, args, module, cls in strategies:
        if condition(yolo_model):
            return deps, args, module, cls
    
    raise ValueError(f"No detector found for model: {yolo_model}")

When the pipeline receives a path like weights/mydetector.pt, the is_mydet_model check returns True, causing get_yolo_inferer to import boxmot.detectors.mydetector.MyDetector automatically.

Step 3: Run Inference with Your Detector

With your detector registered, instantiate DetectorReIDPipeline from boxmot/engine/inference.py using a model path that contains your marker substring.

from boxmot.engine.inference import DetectorReIDPipeline

# Initialize pipeline with custom detector

pipeline = DetectorReIDPipeline(
    yolo_model_path="weights/mydetector.pt",  # Contains "mydet" marker

    device="cpu",
)

# Single image inference

result = next(pipeline.predict("samples/frame.jpg", stream=False))

# Batch processing

import cv2
images = [cv2.imread(p) for p in image_paths]
batch_results = pipeline.predict_batch(images)

The pipeline treats your custom detector identically to built-in backends, automatically applying ReID models, trackers, and visualization callbacks without additional configuration.

Alternative: Explicit Instantiation Without Markers

For experimental prototypes or when you cannot modify the marker registry, instantiate the detector directly and inject it into the pipeline:

from boxmot.engine.inference import DetectorReIDPipeline
from boxmot.detectors.mydetector import MyDetector

# Load custom detector manually

custom_detector = MyDetector(path="weights/mydetector.pt")

# Create pipeline with placeholder, then override

pipeline = DetectorReIDPipeline(
    yolo_model_path="placeholder.pt",
    device="cpu",
)
pipeline.yolo.model = custom_detector

This bypasses the auto-selection logic in get_yolo_inferer while maintaining full pipeline functionality.

Key Source Files and Their Roles

• boxmot/detectors/detector.py: Abstract base class (Detector) defining the __call__(image) contract that all detectors must implement.
• boxmot/detectors/__init__.py: Contains get_yolo_inferer dispatcher and condition functions (is_ultralytics_model, is_yolox_model, etc.) for strategy selection.
• boxmot/engine/inference.py: High-level DetectorReIDPipeline that orchestrates detection, ReID embedding extraction, timing, and callback execution.
• boxmot/detectors/yolox.py, ultralytics.py, rtdetr.py: Reference implementations demonstrating proper detector subclassing patterns.

Summary

• Subclass Detector from boxmot/detectors/detector.py and implement _load_model, preprocess, process, and postprocess methods.
• Return standard format from postprocess: a (N, 6) NumPy array containing [x1, y1, x2, y2, confidence, class_id].
• Register a marker in boxmot/detectors/__init__.py by adding a condition function and strategy tuple to the get_yolo_inferer dispatcher.
• Use standard pipeline by passing a model path containing your marker to DetectorReIDPipeline in boxmot/engine/inference.py.
• Handle dependencies by listing extra pip packages in the strategy tuple if your detector requires libraries beyond the base installation.

Frequently Asked Questions

What output format must my custom detector return?

Your postprocess method must return a NumPy array of shape (N, 6) where each row contains [x1, y1, x2, y2, confidence, class_id]. The DetectorReIDPipeline in boxmot/engine/inference.py calls extract_detections internally, which expects exactly this format to pass detections to the tracking and ReID modules.

Can I use GPU acceleration with custom detectors?

Yes. Move your model to the target device inside _load_model using self.model.to(self.device). The DetectorReIDPipeline passes the device parameter (e.g., "cuda", "cpu", "0") to your detector's constructor, ensuring your model runs on the specified hardware alongside the rest of the pipeline components.

How do I handle extra dependencies for my detector?

Include required pip packages in the second element of your strategy tuple in boxmot/detectors/__init__.py. For example: ("onnxruntime-gpu",) instead of (). The framework's RequirementsChecker (referenced in the loader logic) will automatically install these dependencies when your detector is first imported.

Can I integrate ONNX or TensorRT models?

Absolutely. Modify _load_model to use onnxruntime.InferenceSession for ONNX or torch_tensorrt for TensorRT models. The detector interface is backend-agnostic—you only need to ensure process returns raw tensors compatible with your postprocess logic, and that the final output conforms to the (N, 6) BoxMOT standard format.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}