# How to Configure Multi-Class Object Tracking in ByteTrack

> Learn how to configure multi-class object tracking in ByteTrack. Simply pass detection objects with class identifiers and let ByteTrack handle the rest for seamless tracking across categories.

- Repository: [Roboflow/supervision](https://github.com/roboflow/supervision)
- Tags: how-to-guide
- Published: 2026-04-06

---

**ByteTrack is inherently class-agnostic, meaning you enable multi-class object tracking by simply passing detection objects that include class identifiers—the tracker handles bounding box geometry and confidence scores without internally separating objects by category.**

ByteTrack is a fast, online multi-object tracker widely used in computer vision pipelines. When implemented through the `supervision` library, it processes raw detection tensors to maintain persistent object identities across video frames. Because the tracker focuses exclusively on bounding box coordinates and confidence values in [`src/supervision/tracker/byte_tracker/core.py`](https://github.com/roboflow/supervision/blob/main/src/supervision/tracker/byte_tracker/core.py), it naturally supports multi-class scenarios without requiring class-specific configuration.

## How ByteTrack Handles Classes Internally

The core tracking logic does not reference class labels during the association process. According to the source implementation, the `ByteTrack.__init__` method (lines 43-50) initializes Kalman filter objects and threshold parameters, while the update pipeline processes all detections uniformly regardless of category.

In `ByteTrack.update_with_tensors` (lines 89-100), detections with scores exceeding `track_activation_threshold` are converted to `STrack` objects defined in [`single_object_track.py`](https://github.com/roboflow/supervision/blob/main/single_object_track.py). The Kalman prediction step in `STrack.multi_predict` (lines 63-78) operates purely on bounding box dynamics. The association phase in [`matching.py`](https://github.com/roboflow/supervision/blob/main/matching.py) (lines 44-75) builds cost matrices using IoU distance and fused detection scores, applying the Hungarian algorithm via `matching.linear_assignment` without considering class labels.

The `class_id` field from the `Detections` object is carried through the pipeline and mapped back to output in `ByteTrack.update_with_detections` (lines 28-34), but it never influences track creation or matching decisions in the internal logic.

## Basic Multi-Class Tracking Configuration

To track multiple classes simultaneously, instantiate ByteTrack with your desired thresholds and ensure your detector provides class information in the `Detections` object.

```python
import supervision as sv
from ultralytics import YOLO
import numpy as np

model = YOLO("yolov8s.pt")
tracker = sv.ByteTrack(
    track_activation_threshold=0.25,
    minimum_matching_threshold=0.8,
    minimum_consecutive_frames=1,
    frame_rate=30
)

def process_frame(frame: np.ndarray) -> np.ndarray:
    results = model(frame)[0]
    detections = sv.Detections.from_ultralytics(results)
    detections = tracker.update_with_detections(detections)
    
    labels = [
        f"{results.names[int(cls)]} #{tid}"
        for cls, tid in zip(detections.class_id, detections.tracker_id)
    ]
    
    box_annotator = sv.BoxAnnotator()
    annotated = box_annotator.annotate(frame.copy(), detections)
    return annotated

```

The `update_with_detections` method returns a modified `Detections` object where the `tracker_id` array contains persistent IDs for each bounding box, while the original `class_id` array preserves the category information from your detector. Both arrays maintain index alignment, allowing direct correlation between track IDs and class labels.

## Advanced Multi-Class Workflows

### Building Per-Class Track Collections

Since ByteTrack does not maintain separate buffers for different classes, you must group tracks by class in post-processing. Use the `class_id` and `tracker_id` arrays to organize active tracks into category-specific collections.

```python
from collections import defaultdict

active_tracks_by_class = defaultdict(set)

for cls_id, trk_id in zip(detections.class_id, detections.tracker_id):
    if trk_id != -1:
        active_tracks_by_class[int(cls_id)].add(int(trk_id))

for cls_id, tracks in active_tracks_by_class.items():
    print(f"Class {cls_id}: {len(tracks)} active tracks")

```

This approach leverages the fact that ByteTrack assigns unique tracker IDs across all classes, letting you filter and count per-class statistics after the tracking update completes.

### Configuring Class-Specific Thresholds

For scenarios requiring different tracking parameters per category, instantiate separate `ByteTrack` objects for each class and process detection subsets individually.

```python
trackers = {
    0: sv.ByteTrack(track_activation_threshold=0.3),  # pedestrians

    1: sv.ByteTrack(track_activation_threshold=0.2),  # vehicles

}

def process_frame(frame):
    results = model(frame)[0]
    detections = sv.Detections.from_ultralytics(results)
    merged = sv.Detections.empty()
    
    for cls_id, tracker in trackers.items():
        mask = detections.class_id == cls_id
        class_dets = detections[mask]
        tracked = tracker.update_with_detections(class_dets)
        merged = sv.Detections.concatenate([merged, tracked])
    
    return merged

```

Each tracker instance maintains its own internal Kalman filters and track buffers, allowing fine-grained control over activation thresholds and matching behavior for specific object categories.

## Summary

- ByteTrack processes bounding boxes and confidence scores independently of class labels, making it naturally suited for multi-class tracking in `supervision`.
- The `class_id` field in the `Detections` object is preserved through the tracking pipeline in [`core.py`](https://github.com/roboflow/supervision/blob/main/core.py) but does not influence the Kalman prediction or Hungarian matching algorithms in [`single_object_track.py`](https://github.com/roboflow/supervision/blob/main/single_object_track.py) and [`matching.py`](https://github.com/roboflow/supervision/blob/main/matching.py).
- Per-class statistics and filtering must be implemented in post-processing by grouping `tracker_id` values according to their corresponding `class_id`.
- For class-specific tracking behaviors, initialize separate tracker instances with distinct threshold configurations and process each class subset individually.

## Frequently Asked Questions

### Does ByteTrack automatically separate tracks by object class?

No. ByteTrack is class-agnostic and treats all detections uniformly during the association phase in [`matching.py`](https://github.com/roboflow/supervision/blob/main/matching.py). The algorithm uses IoU-based cost matrices and Kalman predictions from [`single_object_track.py`](https://github.com/roboflow/supervision/blob/main/single_object_track.py) without referencing class labels. You must handle class-specific logic after calling `update_with_detections` by filtering the parallel `class_id` and `tracker_id` arrays.

### Where does the tracker store class information in the output?

The `Detections` object maintains parallel numpy arrays where `detections.class_id` contains the original class index from your detector and `detections.tracker_id` contains the assigned track ID from `ByteTrack.update_with_detections`. These arrays are aligned by index, allowing you to correlate every tracked object with its category using standard array indexing operations.

### Can I assign different confidence thresholds to different object classes?

Yes, but this requires running multiple tracker instances. Because `track_activation_threshold` is set at initialization in `ByteTrack.__init__`, you cannot dynamically adjust it per class within a single tracker. Instead, filter detections by class before tracking and route each subset to a dedicated tracker instance with appropriate thresholds, then concatenate the results using `sv.Detections.concatenate`.

### How do I handle occlusions between different object classes?

ByteTrack handles occlusions using the same mechanism for all classes: unmatched tracks are buffered for a configurable number of frames based on `frame_rate` and `minimum_consecutive_frames` parameters. Since the tracker does not distinguish between classes during the matching phase in `matching.linear_assignment`, occlusions between different classes are resolved purely based on IoU overlap and motion prediction from the shared Kalman filter implementation.