How to Configure BotSORT for Tracking with ReID Features: Complete Setup Guide

To configure BotSORT for ReID tracking, supply a pre-trained ReID model via the reid_weights parameter and tune the appearance_thresh and proximity_thresh thresholds to control when appearance embeddings are fused with motion cues during data association.

BotSORT is the appearance-enhanced successor to SORT, combining Kalman filtering with deep ReID embeddings to maintain identities through occlusions and camera motion. In the mikel-brostrom/boxmot repository, enabling and tuning these ReID features requires specific constructor arguments and threshold adjustments that control how the tracker fuses visual similarity with spatial proximity.

Core BotSORT Architecture

Before configuring ReID parameters, understand the three primary components that handle appearance features in the BotSORT pipeline.

BotSort Class Implementation

The main tracker logic resides in boxmot/trackers/botsort/botsort.py. This class initializes the Kalman filter, manages the camera motion compensation (self.cmc), and orchestrates the two-stage data association process. When with_reid=True, the constructor instantiates the feature extraction backend that computes embeddings for each detection.

STrack with Embeddings

Individual track states are managed by STrack in boxmot/trackers/botsort/botsort_track.py. Unlike basic SORT tracks, each STrack instance stores an appearance embedding vector that is updated as new detections are matched, enabling the tracker to re-identify objects after temporary occlusion.

Configuration and Registry

Default hyperparameters are defined in boxmot/configs/trackers/botsort.yaml, while the tracker registry in boxmot/trackers/tracker_zoo.py maps the name "botsort" to the CLI interface. The ReidAutoBackend class in boxmot/reid/core/auto_backend.py abstracts model loading and feature extraction for supported ReID architectures like OSNet.

Enabling ReID in BotSORT

Activating appearance-based tracking requires passing a valid ReID model path and device configuration to the BotSort constructor.

Required Constructor Arguments

When with_reid=True (the default), the tracker creates the ReID model using:

if self.with_reid:
    self.model = ReidAutoBackend(
        weights=reid_weights, device=device, half=half
    ).model

You must provide:

• reid_weights: Path to a pre-trained .pt file (e.g., osnet_x0_25_msmt17.pt). The CLI automatically resolves built-in model names to cache paths.
• device: Torch device for inference, typically torch.device('cuda') for GPU acceleration.
• half: Boolean flag for FP16 inference; set True for GPU speedups and False for CPU stability.

CLI Activation

The simplest way to enable ReID is via the boxmot CLI, which handles device detection and weight downloading:

boxmot track yolov8n osnet_x0_25_msmt17 botsort --source video.mp4

This command automatically sets with_reid=True, loads the OSNet model, and configures the appropriate tensor device.

Tuning ReID Thresholds

Four key parameters in boxmot/configs/trackers/botsort.yaml control how strictly appearance features influence tracking decisions.

appearance_thresh

Default: 0.25

This parameter sets the maximum cosine distance allowed for an embedding match. Increase this value (e.g., to 0.30 or 0.35) to make the tracker more tolerant of appearance variations, though this risks identity switches when objects look similar. Decrease it for stricter visual matching.

proximity_thresh

Default: 0.5

An IoU threshold that acts as a spatial gate. When boxes are farther apart than this IoU distance, the tracker blocks appearance-based matches even if embeddings are similar. Lower this value (e.g., to 0.3) to restrict ReID matching to spatially nearby candidates, or raise it to allow re-identification across larger gaps.

fuse_first_associate

Default: False

When set to True, the first association stage fuses IoU and embedding scores using a fused cost matrix. This is critical for heavily occluded scenes where motion predictions alone fail. Enable this for crowded environments:

boxmot track yolov8n osnet_x0_25_msmt17 botsort --source video.mp4 fuse_first_associate=True

with_reid

Default: True

Set to False to completely disable the appearance branch, reverting BotSORT to pure motion-based tracking. This eliminates ReID computational overhead but sacrifices robustness during occlusion.

CLI Configuration Examples

Override default thresholds directly from the command line using key=value syntax.

High-Confidence Detection Tracking

For scenes with distinct objects and minimal occlusion:

boxmot track yolov8n osnet_x0_25_msmt17 botsort \
    --source path/to/video.mp4 \
    --save \
    track_high_thresh=0.6 \
    track_low_thresh=0.1 \
    new_track_thresh=0.7 \
    appearance_thresh=0.30 \
    proximity_thresh=0.45 \
    fuse_first_associate=True

Webcam Demo with Tuned ReID

For real-time testing with adjusted appearance tolerance:

boxmot track yolov8n osnet_x0_25_msmt17 botsort \
    --source 0 \
    --show \
    appearance_thresh=0.28 \
    fuse_first_associate=True

Python API Implementation

For custom detector integration, instantiate BotSort directly in your inference loop.

Tracker Initialization

from pathlib import Path
import torch
from boxmot.trackers.botsort.botsort import BotSort

tracker = BotSort(
    reid_weights=Path("~/.cache/torch/hub/checkpoints/osnet_x0_25_msmt17.pt").expanduser(),
    device=torch.device("cuda" if torch.cuda.is_available() else "cpu"),
    half=True,
    appearance_thresh=0.30,
    proximity_thresh=0.45,
    fuse_first_associate=True,
    # BaseTracker params like max_age and min_hits use defaults if omitted

)

Processing Loop

The update method expects detections as a NumPy array of shape (N, 5) containing [x1, y1, x2, y2, confidence] and the current frame image:

import cv2
import numpy as np

cap = cv2.VideoCapture("video.mp4")
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Replace with your detector output

    dets = np.array([[50, 60, 200, 250, 0.92]])  # [x1, y1, x2, y2, conf]

    # Update tracks with ReID features

    tracks = tracker.update(dets, frame)  # Returns [x1, y1, x2, y2, id, conf, cls, det_ind]

    
    for trk in tracks:
        x1, y1, x2, y2, tid, conf, cls, det_idx = trk
        cv2.rectangle(frame, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 2)
        cv2.putText(frame, f"ID:{int(tid)}", (int(x1), int(y1)-5),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 255, 0), 2)

    cv2.imshow("BotSORT + ReID", frame)
    if cv2.waitKey(1) == 27:
        break
cap.release()
cv2.destroyAllWindows()

Internally, the update method splits detections into high and low confidence groups, computes embeddings for high-confidence boxes via self.model.get_features, and executes the two-stage association defined in BotSort._first_association and BotSort._second_association.

Summary

• Supply a ReID model: Pass reid_weights to the BotSort constructor or use the CLI with a built-in model name like osnet_x0_25_msmt17.
• Tune appearance_thresh: Control embedding matching strictness (default 0.25) to balance between identity recovery and ID switches.
• Adjust proximity_thresh: Limit appearance matching to spatially nearby detections using the IoU threshold (default 0.5).
• Enable fusion when needed: Set fuse_first_associate=True to combine motion and appearance cues in the first association stage for occluded scenes.
• Disable if unnecessary: Set with_reid=False to revert to motion-only tracking and reduce computational load.

Frequently Asked Questions

What is the difference between BotSORT and SORT?

SORT relies exclusively on motion cues from Kalman filtering and IoU matching, making it prone to identity switches during occlusion. BotSORT adds a ReID branch that computes deep appearance embeddings for each detection, enabling the tracker to re-identify objects after they re-emerge from behind obstacles, significantly improving ID consistency in crowded scenes.

Which ReID model should I use with BotSORT?

The boxmot repository includes built-in support for OSNet variants. For most applications, osnet_x0_25_msmt17 provides a strong balance between accuracy and inference speed. Larger models like osnet_x1_0 offer higher discriminative power but increase latency. All supported models are automatically downloaded via the ReidAutoBackend when referenced by name in the CLI.

How do I disable ReID if I only want motion-based tracking?

Pass with_reid=False in the Python constructor or explicitly set it in your configuration override. This prevents the tracker from loading the ReidAutoBackend and skips all embedding computation, causing BotSORT to rely solely on the Kalman predictor and IoU matching like the original SORT algorithm.

What does the fuse_first_associate parameter do?

When fuse_first_associate=True, the first stage of data association combines IoU distance and embedding cosine distance into a single fused cost matrix. This allows high-confidence detections to be matched based on both spatial proximity and visual appearance simultaneously, rather than sequentially. Enable this parameter when tracking objects in crowded environments where pure motion prediction frequently overlaps multiple candidates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}