How Magic-Trace Implements Breakpoint-Based Snapshotting Using Linux Perf Hardware Breakpoints

Magic-Trace triggers snapshots by installing hardware breakpoints via the Linux perf subsystem, monitoring the resulting file descriptor through an Async-driven read loop that captures timestamps and register state when the target function is executed.

Magic-trace is a time-tracing tool developed by Jane Street that captures detailed execution snapshots of running processes. The breakpoint-based snapshotting mechanism allows users to trigger traces when specific functions are called by leveraging CPU hardware breakpoints rather than software instrumentation. This article examines the implementation details found in the janestreet/magic-trace repository, tracing the path from breakpoint creation through snapshot triggering.

The Architecture of Breakpoint-Based Snapshotting

The foundation of breakpoint-based snapshotting rests on Linux's perf_event_open system call. In src/breakpoint_stubs.c, the magic_breakpoint_create_stub function constructs a perf_event_attr structure configured for hardware breakpoint monitoring.

Configuring the perf_event_attr Structure

The C stub initializes the structure with specific parameters: type is set to PERF_TYPE_BREAKPOINT, bp_type to HW_BREAKPOINT_X (indicating an execution breakpoint), and bp_addr to the target function address. The sample period is hardcoded to 1, ensuring the breakpoint fires immediately upon execution. The code in src/breakpoint_stubs.c lines 69-86 handles the sys_perf_event_open call and memory-maps a ring buffer for sample delivery, wrapping the resulting file descriptor and mmap region in a custom OCaml value of type Breakpoint.t (src/breakpoint.ml lines 5-10).

Enabling Single-Shot vs. Continuous Breakpoints

After creation, the breakpoint remains inactive until Breakpoint.enable invokes magic_breakpoint_enable_stub. For single-hit scenarios (the default when not using --multi-snapshot), the stub issues ioctl(fd, PERF_EVENT_IOC_REFRESH, 1), which configures the breakpoint to fire exactly once before automatically disabling. Continuous monitoring uses PERF_EVENT_IOC_ENABLE instead. This logic appears in src/breakpoint_stubs.c lines 123-131, returning error codes to the OCaml layer for failure handling.

Async-Driven Breakpoint Monitoring

With the breakpoint enabled, magic-trace must asynchronously detect when the hardware breakpoint triggers. This coordination happens in src/trace.ml, where the breakpoint file descriptor integrates with Jane Street's Async library.

Registering the File Descriptor with Async

The code converts the raw perf file descriptor into an Async_unix.Fd using interruptible_every_ready_to. This registers a callback that fires whenever the file descriptor becomes readable, creating a non-blocking read loop that repeatedly invokes Breakpoint.next_hit. For single-hit breakpoints, the callback automatically disables monitoring after the first sample to prevent redundant snapshot attempts (see src/trace.ml lines 99-108).

Reading Samples from the Ring Buffer

When the callback fires, magic_breakpoint_next_stub (called via Breakpoint.next_hit) walks the memory-mapped ring buffer searching for PERF_RECORD_SAMPLE events. The stub extracts critical data including the timestamp, TSC-derived time, instruction pointer, thread ID, and any user-provided register values. It returns an OCaml Some Hit.t record containing these details, or None if the ring buffer contains no new samples. This implementation spans src/breakpoint_stubs.c lines 61-90 and the OCaml wrapper in src/breakpoint.ml lines 23-26.

From Breakpoint Hit to Snapshot Execution

The transition from detecting a breakpoint to capturing a trace involves coordination between the breakpoint monitoring logic and the backend recording system.

Triggering the Snapshot

Upon receiving a valid Hit.t, trace.ml executes take_snapshot_on_hit, which delegates to Backend.Recording.maybe_take_snapshot to perform the actual trace capture. The implementation stores the hit details for diagnostic purposes and, unless running in multi-snapshot mode, immediately disables further breakpoint monitoring. This critical path appears in src/trace.ml around lines 383-390.

Complete Workflow: From -trigger to Snapshot

Integrating these components reveals the end-to-end path users trigger when specifying -trigger <symbol> on the command line.

Symbol Resolution and Breakpoint Setup

The When_to_snapshot module parses the trigger specification and resolves the target symbol to a concrete memory address. It passes this address to Breakpoint.breakpoint_fd, initiating the creation sequence described above. In src/when_to_snapshot.ml, this logic drives the initialization of the breakpoint infrastructure before the trace begins.

Practical Implementation Example

The following OCaml code demonstrates how magic-trace programmatically establishes a breakpoint-based snapshot trigger:

(* Example: take a snapshot whenever the function `my_func` is called *)

let trigger = Symbol_selection.of_command_string "my_func"
let when_to_snapshot = When_to_snapshot.Application_calls_a_function trigger

let () =
  (* `head_pid` is the pid of the traced process, obtained earlier *)
  let addr = Symbol_selection.addr_of trigger in
  match Breakpoint.breakpoint_fd head_pid ~addr with
  | Ok bp ->
      (* Enable a single‑hit breakpoint (default for non‑multi‑snapshot mode) *)
      Breakpoint.enable bp ~single_hit:true |> Or_error.ok_exn;
      (* The async loop in `trace.ml` will now invoke `take_snapshot_on_hit`
         each time `my_func` is entered. *)
      ()
  | Error err -> failwith (Error.to_string_hum err)

This sequence matches the internal implementation: resolve the address, create the breakpoint via the C stubs, enable it with single-hit semantics, and allow the Async monitor to bridge hardware events to snapshot operations.

Summary

• Hardware breakpoints use the Linux perf subsystem via sys_perf_event_open with PERF_TYPE_BREAKPOINT and HW_BREAKPOINT_X configurations.
• Breakpoint lifecycle moves through creation in src/breakpoint_stubs.c, enabling via ioctl commands, and monitoring through Async_unix.Fd callbacks in src/trace.ml.
• Sample extraction reads PERF_RECORD_SAMPLE events from a memory-mapped ring buffer, yielding timestamps, register values, and thread IDs.
• Snapshot triggering occurs in take_snapshot_on_hit, which delegates to the backend recording system and handles single-shot vs. multi-snapshot modes.
• User interface connects via the -trigger option, processed by When_to_snapshot to drive the entire breakpoint initialization chain.

Frequently Asked Questions

What is breakpoint-based snapshotting in magic-trace?

Breakpoint-based snapshotting is a tracing mechanism that captures program execution snapshots when a specific function or memory address is reached. Unlike software instrumentation, this approach uses CPU hardware breakpoints, allowing zero-overhead monitoring until the exact moment the target code executes, at which point magic-trace captures the full trace buffer.

How does magic-trace use the Linux perf subsystem for breakpoints?

Magic-trace configures hardware breakpoints by calling sys_perf_event_open with a perf_event_attr structure set to PERF_TYPE_BREAKPOINT type and HW_BREAKPOINT_X (execution) monitoring. The C implementation in src/breakpoint_stubs.c manages the resulting file descriptor and memory-maps a ring buffer to receive PERF_RECORD_SAMPLE events when the CPU hits the breakpoint address.

What happens when a breakpoint is hit in magic-trace?

When the CPU triggers a hardware breakpoint, the kernel writes a sample record to the memory-mapped ring buffer. The Async monitor in src/trace.ml detects the ready file descriptor, invokes Breakpoint.next_hit to parse the sample (extracting timestamp, instruction pointer, and register data), and calls take_snapshot_on_hit to capture the trace via Backend.Recording.maybe_take_snapshot.

Can magic-trace trigger multiple snapshots from a single breakpoint?

Yes, when running in multi-snapshot mode (enabled via the --multi-snapshot flag), the breakpoint uses PERF_EVENT_IOC_ENABLE instead of PERF_EVENT_IOC_REFRESH, allowing continuous triggering. In this mode, take_snapshot_on_hit does not disable the breakpoint after the first hit, enabling repeated snapshot captures every time the target function executes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}