# How Magic-Trace Implements Breakpoint-Based Snapshotting Using Linux Perf Hardware Breakpoints

> Discover how Magic-Trace uses Linux perf hardware breakpoints to implement efficient breakpoint-based snapshotting, capturing timestamps and register state on function execution.

- Repository: [Jane Street/magic-trace](https://github.com/janestreet/magic-trace)
- Tags: internals
- Published: 2026-05-24

---

**Magic-Trace triggers snapshots by installing hardware breakpoints via the Linux perf subsystem, monitoring the resulting file descriptor through an Async-driven read loop that captures timestamps and register state when the target function is executed.**

Magic-trace is a time-tracing tool developed by Jane Street that captures detailed execution snapshots of running processes. The **breakpoint-based snapshotting mechanism** allows users to trigger traces when specific functions are called by leveraging CPU hardware breakpoints rather than software instrumentation. This article examines the implementation details found in the `janestreet/magic-trace` repository, tracing the path from breakpoint creation through snapshot triggering.

## The Architecture of Breakpoint-Based Snapshotting

The foundation of breakpoint-based snapshotting rests on Linux's `perf_event_open` system call. In [`src/breakpoint_stubs.c`](https://github.com/janestreet/magic-trace/blob/main/src/breakpoint_stubs.c), the `magic_breakpoint_create_stub` function constructs a `perf_event_attr` structure configured for hardware breakpoint monitoring.

### Configuring the perf_event_attr Structure

The C stub initializes the structure with specific parameters: `type` is set to `PERF_TYPE_BREAKPOINT`, `bp_type` to `HW_BREAKPOINT_X` (indicating an execution breakpoint), and `bp_addr` to the target function address. The sample period is hardcoded to 1, ensuring the breakpoint fires immediately upon execution. The code in [`src/breakpoint_stubs.c`](https://github.com/janestreet/magic-trace/blob/main/src/breakpoint_stubs.c) lines 69-86 handles the `sys_perf_event_open` call and memory-maps a ring buffer for sample delivery, wrapping the resulting file descriptor and mmap region in a custom OCaml value of type `Breakpoint.t` ([`src/breakpoint.ml`](https://github.com/janestreet/magic-trace/blob/main/src/breakpoint.ml) lines 5-10).

### Enabling Single-Shot vs. Continuous Breakpoints

After creation, the breakpoint remains inactive until `Breakpoint.enable` invokes `magic_breakpoint_enable_stub`. For single-hit scenarios (the default when not using `--multi-snapshot`), the stub issues `ioctl(fd, PERF_EVENT_IOC_REFRESH, 1)`, which configures the breakpoint to fire exactly once before automatically disabling. Continuous monitoring uses `PERF_EVENT_IOC_ENABLE` instead. This logic appears in [`src/breakpoint_stubs.c`](https://github.com/janestreet/magic-trace/blob/main/src/breakpoint_stubs.c) lines 123-131, returning error codes to the OCaml layer for failure handling.

## Async-Driven Breakpoint Monitoring

With the breakpoint enabled, magic-trace must asynchronously detect when the hardware breakpoint triggers. This coordination happens in [`src/trace.ml`](https://github.com/janestreet/magic-trace/blob/main/src/trace.ml), where the breakpoint file descriptor integrates with Jane Street's Async library.

### Registering the File Descriptor with Async

The code converts the raw perf file descriptor into an `Async_unix.Fd` using `interruptible_every_ready_to`. This registers a callback that fires whenever the file descriptor becomes readable, creating a non-blocking read loop that repeatedly invokes `Breakpoint.next_hit`. For single-hit breakpoints, the callback automatically disables monitoring after the first sample to prevent redundant snapshot attempts (see [`src/trace.ml`](https://github.com/janestreet/magic-trace/blob/main/src/trace.ml) lines 99-108).

### Reading Samples from the Ring Buffer

When the callback fires, `magic_breakpoint_next_stub` (called via `Breakpoint.next_hit`) walks the memory-mapped ring buffer searching for `PERF_RECORD_SAMPLE` events. The stub extracts critical data including the timestamp, TSC-derived time, instruction pointer, thread ID, and any user-provided register values. It returns an OCaml `Some Hit.t` record containing these details, or `None` if the ring buffer contains no new samples. This implementation spans [`src/breakpoint_stubs.c`](https://github.com/janestreet/magic-trace/blob/main/src/breakpoint_stubs.c) lines 61-90 and the OCaml wrapper in [`src/breakpoint.ml`](https://github.com/janestreet/magic-trace/blob/main/src/breakpoint.ml) lines 23-26.

## From Breakpoint Hit to Snapshot Execution

The transition from detecting a breakpoint to capturing a trace involves coordination between the breakpoint monitoring logic and the backend recording system.

### Triggering the Snapshot

Upon receiving a valid `Hit.t`, [`trace.ml`](https://github.com/janestreet/magic-trace/blob/main/trace.ml) executes `take_snapshot_on_hit`, which delegates to `Backend.Recording.maybe_take_snapshot` to perform the actual trace capture. The implementation stores the hit details for diagnostic purposes and, unless running in multi-snapshot mode, immediately disables further breakpoint monitoring. This critical path appears in [`src/trace.ml`](https://github.com/janestreet/magic-trace/blob/main/src/trace.ml) around lines 383-390.

## Complete Workflow: From -trigger to Snapshot

Integrating these components reveals the end-to-end path users trigger when specifying `-trigger <symbol>` on the command line.

### Symbol Resolution and Breakpoint Setup

The `When_to_snapshot` module parses the trigger specification and resolves the target symbol to a concrete memory address. It passes this address to `Breakpoint.breakpoint_fd`, initiating the creation sequence described above. In [`src/when_to_snapshot.ml`](https://github.com/janestreet/magic-trace/blob/main/src/when_to_snapshot.ml), this logic drives the initialization of the breakpoint infrastructure before the trace begins.

### Practical Implementation Example

The following OCaml code demonstrates how magic-trace programmatically establishes a breakpoint-based snapshot trigger:

```ocaml
(* Example: take a snapshot whenever the function `my_func` is called *)

let trigger = Symbol_selection.of_command_string "my_func"
let when_to_snapshot = When_to_snapshot.Application_calls_a_function trigger

let () =
  (* `head_pid` is the pid of the traced process, obtained earlier *)
  let addr = Symbol_selection.addr_of trigger in
  match Breakpoint.breakpoint_fd head_pid ~addr with
  | Ok bp ->
      (* Enable a single‑hit breakpoint (default for non‑multi‑snapshot mode) *)
      Breakpoint.enable bp ~single_hit:true |> Or_error.ok_exn;
      (* The async loop in `trace.ml` will now invoke `take_snapshot_on_hit`
         each time `my_func` is entered. *)
      ()
  | Error err -> failwith (Error.to_string_hum err)

```

This sequence matches the internal implementation: resolve the address, create the breakpoint via the C stubs, enable it with single-hit semantics, and allow the Async monitor to bridge hardware events to snapshot operations.

## Summary

- **Hardware breakpoints** use the Linux perf subsystem via `sys_perf_event_open` with `PERF_TYPE_BREAKPOINT` and `HW_BREAKPOINT_X` configurations.
- **Breakpoint lifecycle** moves through creation in [`src/breakpoint_stubs.c`](https://github.com/janestreet/magic-trace/blob/main/src/breakpoint_stubs.c), enabling via `ioctl` commands, and monitoring through `Async_unix.Fd` callbacks in [`src/trace.ml`](https://github.com/janestreet/magic-trace/blob/main/src/trace.ml).
- **Sample extraction** reads `PERF_RECORD_SAMPLE` events from a memory-mapped ring buffer, yielding timestamps, register values, and thread IDs.
- **Snapshot triggering** occurs in `take_snapshot_on_hit`, which delegates to the backend recording system and handles single-shot vs. multi-snapshot modes.
- **User interface** connects via the `-trigger` option, processed by `When_to_snapshot` to drive the entire breakpoint initialization chain.

## Frequently Asked Questions

### What is breakpoint-based snapshotting in magic-trace?

**Breakpoint-based snapshotting** is a tracing mechanism that captures program execution snapshots when a specific function or memory address is reached. Unlike software instrumentation, this approach uses CPU hardware breakpoints, allowing zero-overhead monitoring until the exact moment the target code executes, at which point magic-trace captures the full trace buffer.

### How does magic-trace use the Linux perf subsystem for breakpoints?

Magic-trace configures hardware breakpoints by calling `sys_perf_event_open` with a `perf_event_attr` structure set to `PERF_TYPE_BREAKPOINT` type and `HW_BREAKPOINT_X` (execution) monitoring. The C implementation in [`src/breakpoint_stubs.c`](https://github.com/janestreet/magic-trace/blob/main/src/breakpoint_stubs.c) manages the resulting file descriptor and memory-maps a ring buffer to receive `PERF_RECORD_SAMPLE` events when the CPU hits the breakpoint address.

### What happens when a breakpoint is hit in magic-trace?

When the CPU triggers a hardware breakpoint, the kernel writes a sample record to the memory-mapped ring buffer. The Async monitor in [`src/trace.ml`](https://github.com/janestreet/magic-trace/blob/main/src/trace.ml) detects the ready file descriptor, invokes `Breakpoint.next_hit` to parse the sample (extracting timestamp, instruction pointer, and register data), and calls `take_snapshot_on_hit` to capture the trace via `Backend.Recording.maybe_take_snapshot`.

### Can magic-trace trigger multiple snapshots from a single breakpoint?

Yes, when running in multi-snapshot mode (enabled via the `--multi-snapshot` flag), the breakpoint uses `PERF_EVENT_IOC_ENABLE` instead of `PERF_EVENT_IOC_REFRESH`, allowing continuous triggering. In this mode, `take_snapshot_on_hit` does not disable the breakpoint after the first hit, enabling repeated snapshot captures every time the target function executes.