# How to Implement Camera Motion Compensation (CMC) in BoxMOT

> Implement camera motion compensation CMC in BoxMOT for accurate object tracking. Learn how to use the create_cmc factory and integrate CMC into trackers like HybridSort.

- Repository: [Mike/boxmot](https://github.com/mikel-brostrom/boxmot)
- Tags: how-to-guide
- Published: 2026-03-07

---

**BoxMOT provides an extensible camera motion compensation framework that computes affine or homography transforms between frames to correct tracked object positions, accessible via the `create_cmc` factory and integrated into trackers like HybridSort.**

BoxMOT is a popular open-source multi-object tracking library that includes a robust camera motion compensation (CMC) subsystem to stabilize tracking sequences with moving cameras. The CMC implementation uses image-to-image transformations to correct for global camera motion between consecutive frames. This article explains how to implement and configure **camera motion compensation (CMC) in BoxMOT** using the abstract `BaseCMC` interface and concrete algorithms like ECC, SOF, SIFT, and ORB.

## Understanding the CMC Architecture

BoxMOT organizes camera motion compensation around four core components that work together to estimate and apply camera transformations.

### The BaseCMC Interface

The `BaseCMC` abstract class in [`boxmot/motion/cmc/base_cmc.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/base_cmc.py) defines the contract that all CMC methods must implement. It provides the common interface method `apply(img, dets) → warp matrix` and handles preprocessing utilities like mask generation to exclude detections from motion estimation.

### Concrete CMC Implementations

BoxMOT ships with four built-in CMC algorithms, each suited for different motion scenarios:

- **ECC** ([`boxmot/motion/cmc/ecc.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/ecc.py)): Uses OpenCV's ECC (Enhanced Correlation Coefficient) optimizer for intensity-based image alignment
- **SOF** ([`boxmot/motion/cmc/sof.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/sof.py)): Implements sparse optical flow using `goodFeaturesToTrack` and Lucas-Kanade tracking
- **SIFT** ([`boxmot/motion/cmc/sift.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/sift.py)): Feature-based matching using SIFT keypoints with RANSAC-based affine estimation
- **ORB** ([`boxmot/motion/cmc/orb.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/orb.py)): Fast binary descriptor matching using ORB features

### The CMC Registry

The [`boxmot/motion/cmc/__init__.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/__init__.py) file maintains a lazy-loading registry that maps user-friendly method names (e.g., `"ecc"`, `"sof"`) to their respective classes. The `create_cmc` factory function uses this registry to instantiate the requested CMC method with the appropriate parameters.

### Tracker Integration

Most BoxMOT trackers (such as `HybridSort` in [`boxmot/trackers/hybridsort/hybridsort.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/trackers/hybridsort/hybridsort.py)) obtain CMC instances via `get_cmc_method` and call the `apply` method on each frame. The resulting warp matrix is then propagated to every active track through `KalmanBoxTracker.camera_update`, which transforms the bounding box coordinates before data association.

## How to Implement CMC in Your Pipeline

### Stand-Alone CMC Usage

For custom tracking pipelines, you can instantiate and use CMC independently of the built-in trackers. This example demonstrates ECC-based motion compensation:

```python
import cv2
import numpy as np
from boxmot.motion.cmc import create_cmc

# Initialize CMC (ECC, affine mode, downscale to 15% of original size)

cmc = create_cmc(
    "ecc",
    warp_mode=cv2.MOTION_AFFINE,
    eps=1e-5,
    max_iter=100,
    scale=0.15,
    align=False,
)

# Dummy detection array: N×5 (x1, y1, x2, y2, conf)

detections = np.array([[100, 50, 200, 150, 0.9],
                      [300, 80, 380, 160, 0.8]], dtype=np.float32)

# Process a video stream

cap = cv2.VideoCapture("my_video.mp4")

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Compute warp matrix for the current frame

    warp = cmc.apply(frame, detections)

    # Example: compensate the first detection's corners

    x1, y1, x2, y2, _ = detections[0]
    pts = np.array([[x1, y1, 1], [x2, y2, 1]], dtype=np.float32).T
    pts_warped = (warp @ pts)[:2].T
    
    cv2.rectangle(frame,
                  (int(pts_warped[0, 0]), int(pts_warped[0, 1])),
                  (int(pts_warped[1, 0]), int(pts_warped[1, 1])),
                  (0, 255, 0), 2)

    cv2.imshow("CMC compensated", frame)
    if cv2.waitKey(1) == 27:
        break

cap.release()
cv2.destroyAllWindows()

```

The `apply` method receives the current BGR frame and detection array, automatically generates a mask to exclude detected objects from motion estimation, and returns a 2×3 affine or 3×3 homography matrix.

### Enabling CMC Inside BoxMOT Trackers

To enable camera motion compensation in existing trackers like HybridSort, pass the `cmc_method` parameter during initialization:

```python
import torch
import cv2
from pathlib import Path
from boxmot.trackers.hybridsort.hybridsort import HybridSort

# Build tracker with ECC-based CMC

tracker = HybridSort(
    reid_weights=Path("weights/osnet_x0_25_msmt17.pt"),
    device=torch.device("cpu"),
    half=False,
    cmc_method="ecc",               # Select CMC method here

    warp_mode=cv2.MOTION_AFFINE,    # Specific to ECC; ignored by other methods

    low_thresh=0.1,
    delta_t=3,
    inertia=0.05,
)

# Run tracking

video_path = "demo.mp4"
output = tracker.run(video_path)

```

Under the hood, `HybridSort.__init__` calls `get_cmc_method(cmc_method)()` (line 46 in [`hybridsort.py`](https://github.com/mikel-brostrom/boxmot/blob/main/hybridsort.py)). During each update cycle, the tracker computes `warp = self.cmc.apply(img, dets_idx)` (line 90), then applies the warp to all active tracks via `trk.camera_update(warp)` (line 94) before the association step.

## Choosing the Right CMC Method

Select a CMC algorithm based on your scene characteristics and computational constraints:

| Method | Best For | Strengths | Limitations |
|--------|----------|-----------|-------------|
| **ECC** | Smooth translational/rotational motion | Fast, works on intensity images without keypoints | May fail on low-texture frames; falls back to identity |
| **SOF** | Moderate motion with trackable corners | Robust to small rotations; handles outliers via RANSAC | Requires sufficient good features; slower than ECC |
| **SIFT** | Large viewpoint/scale changes | Highly accurate with enough keypoints | Higher computational cost; struggles on low-resolution frames |
| **ORB** | Real-time applications needing scale invariance | Faster than SIFT; binary descriptors | Less accurate than SIFT on challenging textures |

All methods support the `scale` parameter (default 0.15) to downsample images before processing, with translation components up-scaled back to original resolution after estimation (see lines 80-84 in [`ecc.py`](https://github.com/mikel-brostrom/boxmot/blob/main/ecc.py)).

## Implementing Custom CMC Methods

To add a proprietary motion estimator, subclass `BaseCMC` and register it in the CMC registry:

```python

# my_cmc.py

from boxmot.motion.cmc.base_cmc import BaseCMC
import numpy as np

class MyCMC(BaseCMC):
    def __init__(self, scale: float = 0.2):
        super().__init__()
        self.scale = scale
        self.grayscale = True

    def apply(self, img: np.ndarray, dets: np.ndarray = None) -> np.ndarray:
        # Custom motion estimation logic here

        return np.eye(2, 3, dtype=np.float32)

```

Register the class in [`boxmot/motion/cmc/__init__.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/__init__.py):

```python
from .my_cmc import MyCMC
_CMC_REGISTRY["my_cmc"] = _LazyLoader("boxmot.motion.cmc.my_cmc", "MyCMC")

```

You can now instantiate it via `create_cmc("my_cmc")` or pass `cmc_method="my_cmc"` to any BoxMOT tracker.

## Key Source Files

| Path | Description |
|------|-------------|
| [`boxmot/motion/cmc/base_cmc.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/base_cmc.py) | Abstract base class defining the CMC contract (`apply`, preprocessing, mask generation) |
| [`boxmot/motion/cmc/ecc.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/ecc.py) | ECC-based implementation using OpenCV's ECC optimizer |
| [`boxmot/motion/cmc/sof.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/sof.py) | Sparse optical flow using Lucas-Kanade |
| [`boxmot/motion/cmc/sift.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/sift.py) | SIFT feature-matching implementation |
| [`boxmot/motion/cmc/orb.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/orb.py) | ORB binary descriptor implementation |
| [`boxmot/motion/cmc/__init__.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/__init__.py) | Lazy-loading registry mapping method names to classes |
| [`boxmot/trackers/hybridsort/hybridsort.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/trackers/hybridsort/hybridsort.py) | Reference implementation showing CMC integration in a tracker |

## Summary

- **BoxMOT's CMC framework** centers on the `BaseCMC` interface and the `create_cmc` factory for instantiating motion compensators
- **Four built-in methods** (ECC, SOF, SIFT, ORB) handle different camera motion scenarios, selectable via string names in the registry
- **Integration requires** calling `cmc.apply()` to get the warp matrix, then `tracker.camera_update(warp)` to correct Kalman filter states
- **Performance tuning** is available through the `scale` parameter for downsampling and method-specific arguments like `warp_mode`
- **Extensibility** is supported by subclassing `BaseCMC` and registering custom implementations in [`boxmot/motion/cmc/__init__.py`](https://github.com/mikel-brostrom/boxmot/blob/main/boxmot/motion/cmc/__init__.py)

## Frequently Asked Questions

### What is the difference between ECC and SOF camera motion compensation?

**ECC (Enhanced Correlation Coefficient)** performs intensity-based image alignment using optimization over pixel values, making it fast and effective for smooth camera movements without requiring explicit feature detection. **SOF (Sparse Optical Flow)** tracks specific corner points between frames using the Lucas-Kanade algorithm, which is more robust to moderate rotations and independent object motion but requires sufficient textured features in the scene. ECC is generally preferred for real-time applications with stable lighting, while SOF handles dynamic scenes better.

### How does the `scale` parameter affect CMC performance?

The `scale` parameter (default 0.15) resizes the input image to the specified fraction of its original dimensions before motion estimation, significantly reducing computational cost. After estimating the transform on the downsampled image, the translation components are automatically up-scaled back to original coordinates (as implemented in lines 80-84 of [`ecc.py`](https://github.com/mikel-brostrom/boxmot/blob/main/ecc.py)). Lower values increase speed but may reduce accuracy on fine-grained motion; values closer to 1.0 improve precision at the cost of processing time.

### Can I use camera motion compensation with any BoxMOT tracker?

Most modern BoxMOT trackers support CMC through the `cmc_method` initialization argument, including HybridSort and BotSort variants. The tracker internally calls `get_cmc_method()` to instantiate the requested CMC class and applies the warp matrix to all active tracks before the data association step. Check the specific tracker's `__init__` signature in `boxmot/trackers/` to confirm CMC support and available method-specific parameters.

### Why does the CMC implementation exclude detections from motion estimation?

The `generate_mask` method in `BaseCMC` creates a binary mask that removes detection regions from the image area used for motion estimation. This prevents moving objects from biasing the global camera transform calculation, ensuring that the estimated warp represents only camera motion rather than object motion. The mask is automatically applied when `apply(img, dets)` receives a non-empty detection array.