# OpenPilot Route Replay Architecture: Deep Dive into the Debugging Implementation

> Explore the OpenPilot route replay architecture a deep dive into its C++ multi-threaded subsystem for debugging recorded drives. Learn about segment loading event streaming and vision frame serving.

- Repository: [comma.ai/openpilot](https://github.com/commaai/openpilot)
- Tags: deep-dive
- Published: 2026-03-05

---

**The OpenPilot route replay functionality is implemented as a multi-threaded C++ subsystem centered around the `Replay` class that orchestrates segment loading, event streaming, and vision frame serving to reconstruct recorded drives for debugging, using `SegmentManager` for asynchronous data fetching, `CameraServer` for VisionIPC frame distribution, and a dedicated streaming thread with precise speed control and seeking capabilities.**

The route replay functionality in the [commaai/openpilot](https://github.com/commaai/openpilot) repository provides developers with a deterministic way to reproduce recorded drives for testing and debugging. This self-contained tool reads compressed route data, reconstructs the original CAN bus traffic and vision streams, and feeds them into the same processing pipelines used during live driving. Understanding this architecture is essential for extending debugging capabilities or integrating replay-based regression testing into development workflows.

## Core Architecture Components

The route replay system follows a layered design where each component handles a specific aspect of data ingestion, synchronization, or distribution.

### The Replay Orchestrator Class

The [`Replay`](https://github.com/commaai/openpilot/blob/master/tools/replay/replay.h) class serves as the primary facade, defined in [`tools/replay/replay.h`](https://github.com/commaai/openpilot/blob/main/tools/replay/replay.h) and implemented in `tools/replay/replay.cc`. It manages the entire lifecycle from route loading to event publication, exposing methods like `load()`, `start()`, and `waitForFinished()` for external control.

Key member variables include `route_` for route identification, `segment_manager_` for data acquisition, and `stream_thread_` for background processing. The class maintains shared state through atomic variables such as `current_segment_`, `cur_mono_time_`, and `seeking_to_`, which enable thread-safe seeking and speed adjustments during active replay.

### Segment Management and Caching

The [`SegmentManager`](https://github.com/commaai/openpilot/blob/master/tools/replay/seg_mgr.h) class, declared in [`tools/replay/seg_mgr.h`](https://github.com/commaai/openpilot/blob/main/tools/replay/seg_mgr.h) and implemented in `tools/replay/seg_mgr.cc`, handles the asynchronous downloading and caching of per-minute log segments. It manages `.qlog` files and associated vision frames, merging newly available segments into a unified timeline through the `onSegmentsMerged` callback registered during `setupSegmentManager()`.

When filters are specified, the segment manager selectively retains only required services, reducing memory footprint. It provides the `EventData` structure containing sorted events and per-segment frame maps that the streaming thread consumes.

### Timeline and Event Indexing

The [`Timeline`](https://github.com/commaai/openpilot/blob/master/tools/replay/timeline.h) class constructs a lightweight index of high-level events from `drive_event` messages, including alerts and control state changes. Its `initialize()` method processes the route's event stream to build searchable entries, while `findAlertAtTime()` enables precise navigation to specific warnings.

This indexing supports the `seekToFlag()` functionality, allowing developers to jump directly to critical moments such as disengagements or system alerts without manually scrubbing through timestamps.

### Vision Frame Serving

The [`CameraServer`](https://github.com/commaai/openpilot/blob/master/tools/replay/camera.h) class manages VisionIPC sockets for distributing decoded frames to the UI and other consumers. It handles three camera types—RoadCam, DriverCam, and WideRoadCam—pushing frames through the same VisionIPC mechanism used during live driving.

Initialization occurs in `startStream()` only when the `REPLAY_FLAG_NO_VIPC` flag is absent, ensuring vision replay can be disabled for headless testing scenarios.

## Data Flow and Execution Model

The route replay functionality processes data through distinct phases, from initial route parsing to continuous event streaming.

### Initialization and Route Loading

The execution begins with `Replay::load()`, which triggers `segment_manager_->setup()` and initiates metadata fetching. The system calculates temporal boundaries via `min_seconds_` and `max_seconds_` to establish the replay timeline. 

During `start()`, the system calls `startStream()`, which locates the first segment's `INIT_DATA` event to extract wall-clock timing and persists `CAR_PARAMS` into the Params store for controller compatibility. The `Timeline::initialize()` method concurrently builds the event index for alert navigation.

### The Streaming Thread Loop

The [`streamThread()`](https://github.com/commaai/openpilot/blob/master/tools/replay/replay.cc) method runs in a dedicated background thread, implementing the core publishing loop. It waits on `stream_cv_` for the `events_ready_` condition, then processes the unified event list from `event_data_->events`.

For each event, the thread:
1. Calculates relative timing based on `cur_mono_time_` and the configured speed factor via `setSpeed()`
2. Sleeps to maintain real-time pacing (unless `REPLAY_FLAG_BENCHMARK` is active)
3. Dispatches messages through `publishMessage()` for CAN events or `publishFrame()` for vision data
4. Routes data to either the internal `PubMaster` or an externally provided `SubMaster`

### Seeking, Pausing, and Speed Control

Interactive control is implemented through signal-based interruption and atomic state updates. The `interruptStream()` method sends `SIGUSR1` to `stream_thread_`, causing the blocking loop to check updated state variables.

The `seekTo()` method updates `seeking_to_` and `cur_mono_time_`, then triggers the interruption. Upon resuming, the thread re-synchronizes and invokes `onSeekedTo` callbacks for UI updates. Pausing toggles atomic flags that cause the loop to skip event processing while maintaining the thread context.

## Configuration Flags and Modes

The route replay functionality supports multiple operational modes through bitwise flags defined in [`replay.h`](https://github.com/commaai/openpilot/blob/main/replay.h).

### Standard Replay vs. Benchmark Mode

The `REPLAY_FLAG_BENCHMARK` mode transforms the tool into a throughput measurement instrument. When enabled, the streaming thread eliminates sleep calls, records per-segment processing durations, and calculates realtime multiplication factors. Upon completion, the system signals `benchmark_cv_` and prints a concise timeline of segment processing statistics.

Standard replay respects the speed factor configured via `setSpeed()`, defaulting to 1.0x realtime with support for fractional speeds for detailed analysis.

### Service Filtering and VIPC Control

Service filtering operates through constructor parameters `allow_list` and `block_list`, which `setupServices()` processes to determine active ZMQ topics. This selective subscription prevents processing of high-frequency radar or debug messages when unnecessary.

VisionIPC can be disabled entirely using `REPLAY_FLAG_NO_VIPC`, preventing `CameraServer` initialization and reducing resource overhead for pure CAN bus analysis.

## Practical Usage Examples

### Command-Line Interface

Execute the demo route bundled with the repository:

```bash
tools/replay/replay --demo

```

Replay a specific route with benchmark mode enabled:

```bash
tools/replay/replay --benchmark 5beb9b58bd12b691/0000010a--a51155e496

```

### Python Wrapper Integration

The [[`can_replay.py`](https://github.com/commaai/openpilot/blob/main/can_replay.py)](https://github.com/commaai/openpilot/blob/master/tools/replay/can_replay.py) script demonstrates programmatic invocation:

```python
#!/usr/bin/env python3
import subprocess

def replay_route(route_id: str, benchmark: bool = False):
    cmd = ["tools/replay/replay"]
    if benchmark:
        cmd.append("--benchmark")
    cmd.append(route_id)
    subprocess.run(cmd)

# Replay with performance measurement

replay_route("7e0b2d8a7c5e5c99/2022-01-01--00-00-00", benchmark=True)

```

### C++ Library Integration

Embed the replay functionality directly into custom debugging tools:

```cpp
#include "tools/replay/replay.h"

int main() {
  // Allow only CAN and GPS services, enable looping
  std::vector<std::string> allow = {"can", "gpsLocation"};
  Replay replay("5beb9b58bd12b691/0000010a--a51155e496", 
                allow, {}, nullptr, 
                REPLAY_FLAG_NONE, "", true);

  if (!replay.load()) {
    return 1;
  }

  // Install custom event filter
  replay.installEventFilter([](const Event* e) {
    return e->which == cereal::Event::Which::CAN;
  });

  // Configure callbacks and playback
  replay.onSeekedTo = [](double sec) {
    printf("Seeked to %.2f seconds\n", sec);
  };
  
  replay.setSpeed(2.0f);  // 2x speed
  replay.start();
  replay.waitForFinished();
  return 0;
}

```

## Summary

- The **route replay functionality** centers on the [`Replay`](https://github.com/commaai/openpilot/blob/master/tools/replay/replay.h) class, which coordinates segment loading, event streaming, and vision serving through a multi-threaded architecture.
- **SegmentManager** handles asynchronous downloading and caching of per-minute route segments, providing unified event data to the streaming thread.
- A dedicated **streaming thread** processes time-ordered events, supports variable speed playback via `setSpeed()`, and can be interrupted for seeking using `SIGUSR1` signals.
- **CameraServer** distributes decoded vision frames through VisionIPC sockets, while the **Timeline** class indexes drive events for rapid navigation to alerts.
- **Benchmark mode** (`REPLAY_FLAG_BENCHMARK`) enables performance regression testing by eliminating sleep delays and publishing per-segment throughput statistics.

## Frequently Asked Questions

### How does the route replay functionality handle seeking to specific timestamps?

The seeking mechanism uses signal-based interruption and atomic state updates. When `seekTo()` is called, it updates the `seeking_to_` and `cur_mono_time_` atomic variables, then invokes `interruptStream()` which sends `SIGUSR1` to the streaming thread. This causes the blocking `streamThread()` to break from its current sleep, check the updated position variables, and resume processing from the new timestamp, triggering any registered `onSeekedTo` callbacks upon completion.

### What is the purpose of the SegmentManager in the route replay architecture?

**SegmentManager** decouples data acquisition from event processing by managing the asynchronous download and caching of per-minute route segments (`.qlog` files and vision frames). It maintains a local cache of downloaded data, merges newly available segments into a continuous timeline through the `onSegmentsMerged` callback, and optionally filters services to reduce memory usage, allowing the replay system to handle long routes without loading all data into memory simultaneously.

### Can the route replay functionality operate without vision frame processing?

Yes, vision serving is optional and controlled by the `REPLAY_FLAG_NO_VIPC` flag. When this flag is set during `Replay` construction, the `CameraServer` is never instantiated in `startStream()`, and the system operates in a headless mode suitable for pure CAN bus analysis or computational benchmarking without the overhead of VisionIPC socket management and frame decoding.

### How does benchmark mode differ from standard replay operation?

**Benchmark mode** (`REPLAY_FLAG_BENCHMARK`) modifies the streaming thread to eliminate all sleep calls between events, allowing the system to process data as fast as possible. It records the processing time for each segment and calculates realtime multiplication factors (e.g., "5.2× realtime"), printing a summary timeline upon completion. This mode is essential for performance regression testing, as it measures maximum throughput rather than maintaining real-time synchronization.