# How Magic-Trace Manages Multi-Snapshotting with Switch-Output Signals

> Learn how Magic-Trace achieves multi-snapshotting using switch-output signals. Discover seamless output file rotation with SIGUSR2 without interrupting tracing.

- Repository: [Jane Street/magic-trace](https://github.com/janestreet/magic-trace)
- Tags: internals
- Published: 2026-05-24

---

**Magic-Trace enables continuous multi-snapshotting by passing `--switch-output=signal` to `perf record` and forwarding SIGUSR2 to rotate output files without interrupting the tracing session.**

Magic-Trace is a time-travel tracing debugger developed by Jane Street that captures high-resolution execution snapshots using Linux perf. When running in **multi-snapshot mode**, the tool leverages the **`--switch-output=signal`** flag to generate sequential perf data files on demand, allowing developers to capture multiple distinct moments during a single program execution.

## Enabling Multi-Snapshot Mode

To activate multi-snapshotting, invoke `magic-trace` with the `-multi-snapshot true` flag. This boolean value propagates through the backend interface defined in [`src/backend_intf.ml`](https://github.com/janestreet/magic-trace/blob/main/src/backend_intf.ml), ultimately altering how the perf command is constructed and how the tracer manages breakpoint lifetimes.

## Constructing the Perf Command

In [`src/perf_tool_backend.ml`](https://github.com/janestreet/magic-trace/blob/main/src/perf_tool_backend.ml), the function responsible for building the perf invocation checks the `multi_snapshot` configuration. When enabled, it appends `"--switch-output=signal"` to the argument list, instructing perf to listen for a specific signal and write a new snapshot file upon receipt.

The relevant logic appears around lines 502-506:

```ocaml
(* In src/perf_tool_backend.ml – building the perf command *)
let switch_opts =
  match multi_snapshot with
  | true  -> [ "--switch-output=signal" ]   (* <-- enables multi-snapshot *)
  | false -> [] in
let argv = List.concat
  [ [ perf; "record"; "-o"; record_dir ^/ "perf.data"; "--timestamp" ]
  ; event_opts
  ; overwrite_opts
  ; switch_opts                (* this is the key list *)
  ; thread_opts
  ; pid_opt
  ; control_opt
  ; kcore_opts
  ; snapshot_size_opt
  ; Callgraph_mode.to_perf_record_args selected_callgraph_mode
  ] in

```

With this flag, perf generates a sequence of files such as `perf.data`, `perf.data.1`, `perf.data.2`, and so on, each representing an independent snapshot.

## Signal Handling and Forwarding

The implementation relies on **SIGUSR2** as the trigger mechanism. Magic-Trace installs a signal handler to forward this signal to the perf child process, coordinating snapshot rotation without stopping the trace.

### Handler Setup

The `Control` module in [`vendor/tracing/src/tool_output.ml`](https://github.com/janestreet/magic-trace/blob/main/vendor/tracing/src/tool_output.ml) registers the SIGUSR2 handler during initialization. This abstraction provides a clean interface for snapshot management and shutdown procedures.

### Triggering Snapshots Programmatically

When a snapshot condition occurs—whether from a user-triggered `Ctrl-C` or a hit on a specified trigger symbol—the system calls `Control.take_snapshot`. This function sends SIGUSR2 to the perf child PID:

```ocaml
(* In vendor/tracing/src/tool_output.ml – the Control module *)
let take_snapshot control perf_pid =
  (* Forward the signal that perf expects *)
  Core_unix.kill perf_pid Sys.sigusr2

```

Perf receives this signal, closes the current output file, and immediately begins writing to the next incremental file in the sequence.

## Maintaining Breakpoint Persistence

In multi-snapshot mode, the tracer must keep breakpoints active after the first hit to allow subsequent captures. The logic in [`src/trace.ml`](https://github.com/janestreet/magic-trace/blob/main/src/trace.ml) (around lines 389-396) determines whether to disable the breakpoint after a single trigger.

The key variable assignment is:

```ocaml
(* In src/trace.ml – keeping the breakpoint alive for multiple snapshots *)
let single_hit = not opts.multi_snapshot in
if single_hit then Ivar.fill_if_empty done_ivar ()

```

When `multi_snapshot` is `true`, `single_hit` evaluates to `false`, preventing the tracer from filling the completion Ivar and terminating the session. This allows the tracing loop to continue, enabling further snapshots via subsequent SIGUSR2 signals.

## Summary

- **Multi-snapshot mode** is activated via the `-multi-snapshot true` command-line flag.
- The `--switch-output=signal` argument in [`src/perf_tool_backend.ml`](https://github.com/janestreet/magic-trace/blob/main/src/perf_tool_backend.ml) instructs perf to listen for SIGUSR2 to rotate output files.
- The `Control` module in [`vendor/tracing/src/tool_output.ml`](https://github.com/janestreet/magic-trace/blob/main/vendor/tracing/src/tool_output.ml) forwards SIGUSR2 to the perf child process to trigger snapshot writing.
- In [`src/trace.ml`](https://github.com/janestreet/magic-trace/blob/main/src/trace.ml), setting `single_hit = not opts.multi_snapshot` keeps breakpoints active, allowing multiple captures per execution.

## Frequently Asked Questions

### What is the difference between single and multi-snapshot mode in magic-trace?

Single-snapshot mode captures one trace and exits immediately, while multi-snapshot mode keeps the perf process running, allowing you to capture many snapshots during a single execution by sending multiple SIGUSR2 signals.

### Which signal does magic-trace use for switch-output?

Magic-Trace uses **SIGUSR2**. This specific signal is expected by perf when the `--switch-output=signal` flag is provided, causing perf to close the current file and begin writing to a new one.

### How does magic-trace keep the tracer running between snapshots?

The tracer evaluates `let single_hit = not opts.multi_snapshot` in [`src/trace.ml`](https://github.com/janestreet/magic-trace/blob/main/src/trace.ml). When multi-snapshot is enabled, this prevents the automatic shutdown logic from triggering after the first breakpoint hit, keeping the trace session alive for additional snapshots.

### Where is the signal forwarding logic implemented?

The signal forwarding logic resides in [`vendor/tracing/src/tool_output.ml`](https://github.com/janestreet/magic-trace/blob/main/vendor/tracing/src/tool_output.ml) within the `Control` module. This module provides `take_snapshot` and `shutdown` functions that manage SIGUSR2 delivery to the child perf process.