OpenPilot Route Replay Architecture: Deep Dive into the Debugging Implementation

The OpenPilot route replay functionality is implemented as a multi-threaded C++ subsystem centered around the Replay class that orchestrates segment loading, event streaming, and vision frame serving to reconstruct recorded drives for debugging, using SegmentManager for asynchronous data fetching, CameraServer for VisionIPC frame distribution, and a dedicated streaming thread with precise speed control and seeking capabilities.

The route replay functionality in the commaai/openpilot repository provides developers with a deterministic way to reproduce recorded drives for testing and debugging. This self-contained tool reads compressed route data, reconstructs the original CAN bus traffic and vision streams, and feeds them into the same processing pipelines used during live driving. Understanding this architecture is essential for extending debugging capabilities or integrating replay-based regression testing into development workflows.

Core Architecture Components

The route replay system follows a layered design where each component handles a specific aspect of data ingestion, synchronization, or distribution.

The Replay Orchestrator Class

The Replay class serves as the primary facade, defined in tools/replay/replay.h and implemented in tools/replay/replay.cc. It manages the entire lifecycle from route loading to event publication, exposing methods like load(), start(), and waitForFinished() for external control.

Key member variables include route_ for route identification, segment_manager_ for data acquisition, and stream_thread_ for background processing. The class maintains shared state through atomic variables such as current_segment_, cur_mono_time_, and seeking_to_, which enable thread-safe seeking and speed adjustments during active replay.

Segment Management and Caching

The SegmentManager class, declared in tools/replay/seg_mgr.h and implemented in tools/replay/seg_mgr.cc, handles the asynchronous downloading and caching of per-minute log segments. It manages .qlog files and associated vision frames, merging newly available segments into a unified timeline through the onSegmentsMerged callback registered during setupSegmentManager().

When filters are specified, the segment manager selectively retains only required services, reducing memory footprint. It provides the EventData structure containing sorted events and per-segment frame maps that the streaming thread consumes.

Timeline and Event Indexing

The Timeline class constructs a lightweight index of high-level events from drive_event messages, including alerts and control state changes. Its initialize() method processes the route's event stream to build searchable entries, while findAlertAtTime() enables precise navigation to specific warnings.

This indexing supports the seekToFlag() functionality, allowing developers to jump directly to critical moments such as disengagements or system alerts without manually scrubbing through timestamps.

Vision Frame Serving

The CameraServer class manages VisionIPC sockets for distributing decoded frames to the UI and other consumers. It handles three camera types—RoadCam, DriverCam, and WideRoadCam—pushing frames through the same VisionIPC mechanism used during live driving.

Initialization occurs in startStream() only when the REPLAY_FLAG_NO_VIPC flag is absent, ensuring vision replay can be disabled for headless testing scenarios.

Data Flow and Execution Model

The route replay functionality processes data through distinct phases, from initial route parsing to continuous event streaming.

Initialization and Route Loading

The execution begins with Replay::load(), which triggers segment_manager_->setup() and initiates metadata fetching. The system calculates temporal boundaries via min_seconds_ and max_seconds_ to establish the replay timeline.

During start(), the system calls startStream(), which locates the first segment's INIT_DATA event to extract wall-clock timing and persists CAR_PARAMS into the Params store for controller compatibility. The Timeline::initialize() method concurrently builds the event index for alert navigation.

The Streaming Thread Loop

The streamThread() method runs in a dedicated background thread, implementing the core publishing loop. It waits on stream_cv_ for the events_ready_ condition, then processes the unified event list from event_data_->events.

For each event, the thread:

1. Calculates relative timing based on cur_mono_time_ and the configured speed factor via setSpeed()
2. Sleeps to maintain real-time pacing (unless REPLAY_FLAG_BENCHMARK is active)
3. Dispatches messages through publishMessage() for CAN events or publishFrame() for vision data
4. Routes data to either the internal PubMaster or an externally provided SubMaster

Seeking, Pausing, and Speed Control

Interactive control is implemented through signal-based interruption and atomic state updates. The interruptStream() method sends SIGUSR1 to stream_thread_, causing the blocking loop to check updated state variables.

The seekTo() method updates seeking_to_ and cur_mono_time_, then triggers the interruption. Upon resuming, the thread re-synchronizes and invokes onSeekedTo callbacks for UI updates. Pausing toggles atomic flags that cause the loop to skip event processing while maintaining the thread context.

Configuration Flags and Modes

The route replay functionality supports multiple operational modes through bitwise flags defined in replay.h.

Standard Replay vs. Benchmark Mode

The REPLAY_FLAG_BENCHMARK mode transforms the tool into a throughput measurement instrument. When enabled, the streaming thread eliminates sleep calls, records per-segment processing durations, and calculates realtime multiplication factors. Upon completion, the system signals benchmark_cv_ and prints a concise timeline of segment processing statistics.

Standard replay respects the speed factor configured via setSpeed(), defaulting to 1.0x realtime with support for fractional speeds for detailed analysis.

Service Filtering and VIPC Control

Service filtering operates through constructor parameters allow_list and block_list, which setupServices() processes to determine active ZMQ topics. This selective subscription prevents processing of high-frequency radar or debug messages when unnecessary.

VisionIPC can be disabled entirely using REPLAY_FLAG_NO_VIPC, preventing CameraServer initialization and reducing resource overhead for pure CAN bus analysis.

Practical Usage Examples

Command-Line Interface

Execute the demo route bundled with the repository:

tools/replay/replay --demo

Replay a specific route with benchmark mode enabled:

tools/replay/replay --benchmark 5beb9b58bd12b691/0000010a--a51155e496

Python Wrapper Integration

The [can_replay.py](https://github.com/commaai/openpilot/blob/master/tools/replay/can_replay.py) script demonstrates programmatic invocation:

#!/usr/bin/env python3
import subprocess

def replay_route(route_id: str, benchmark: bool = False):
    cmd = ["tools/replay/replay"]
    if benchmark:
        cmd.append("--benchmark")
    cmd.append(route_id)
    subprocess.run(cmd)

# Replay with performance measurement

replay_route("7e0b2d8a7c5e5c99/2022-01-01--00-00-00", benchmark=True)

C++ Library Integration

Embed the replay functionality directly into custom debugging tools:

#include "tools/replay/replay.h"

int main() {
  // Allow only CAN and GPS services, enable looping
  std::vector<std::string> allow = {"can", "gpsLocation"};
  Replay replay("5beb9b58bd12b691/0000010a--a51155e496", 
                allow, {}, nullptr, 
                REPLAY_FLAG_NONE, "", true);

  if (!replay.load()) {
    return 1;
  }

  // Install custom event filter
  replay.installEventFilter([](const Event* e) {
    return e->which == cereal::Event::Which::CAN;
  });

  // Configure callbacks and playback
  replay.onSeekedTo = [](double sec) {
    printf("Seeked to %.2f seconds\n", sec);
  };
  
  replay.setSpeed(2.0f);  // 2x speed
  replay.start();
  replay.waitForFinished();
  return 0;
}

Summary

• The route replay functionality centers on the Replay class, which coordinates segment loading, event streaming, and vision serving through a multi-threaded architecture.
• SegmentManager handles asynchronous downloading and caching of per-minute route segments, providing unified event data to the streaming thread.
• A dedicated streaming thread processes time-ordered events, supports variable speed playback via setSpeed(), and can be interrupted for seeking using SIGUSR1 signals.
• CameraServer distributes decoded vision frames through VisionIPC sockets, while the Timeline class indexes drive events for rapid navigation to alerts.
• Benchmark mode (REPLAY_FLAG_BENCHMARK) enables performance regression testing by eliminating sleep delays and publishing per-segment throughput statistics.

Frequently Asked Questions

How does the route replay functionality handle seeking to specific timestamps?

The seeking mechanism uses signal-based interruption and atomic state updates. When seekTo() is called, it updates the seeking_to_ and cur_mono_time_ atomic variables, then invokes interruptStream() which sends SIGUSR1 to the streaming thread. This causes the blocking streamThread() to break from its current sleep, check the updated position variables, and resume processing from the new timestamp, triggering any registered onSeekedTo callbacks upon completion.

What is the purpose of the SegmentManager in the route replay architecture?

SegmentManager decouples data acquisition from event processing by managing the asynchronous download and caching of per-minute route segments (.qlog files and vision frames). It maintains a local cache of downloaded data, merges newly available segments into a continuous timeline through the onSegmentsMerged callback, and optionally filters services to reduce memory usage, allowing the replay system to handle long routes without loading all data into memory simultaneously.

Can the route replay functionality operate without vision frame processing?

Yes, vision serving is optional and controlled by the REPLAY_FLAG_NO_VIPC flag. When this flag is set during Replay construction, the CameraServer is never instantiated in startStream(), and the system operates in a headless mode suitable for pure CAN bus analysis or computational benchmarking without the overhead of VisionIPC socket management and frame decoding.

How does benchmark mode differ from standard replay operation?

Benchmark mode (REPLAY_FLAG_BENCHMARK) modifies the streaming thread to eliminate all sleep calls between events, allowing the system to process data as fast as possible. It records the processing time for each segment and calculates realtime multiplication factors (e.g., "5.2× realtime"), printing a summary timeline upon completion. This mode is essential for performance regression testing, as it measures maximum throughput rather than maintaining real-time synchronization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}