How Magic-Trace Manages Multi-Snapshotting with Switch-Output Signals

Magic-Trace enables continuous multi-snapshotting by passing --switch-output=signal to perf record and forwarding SIGUSR2 to rotate output files without interrupting the tracing session.

Magic-Trace is a time-travel tracing debugger developed by Jane Street that captures high-resolution execution snapshots using Linux perf. When running in multi-snapshot mode, the tool leverages the --switch-output=signal flag to generate sequential perf data files on demand, allowing developers to capture multiple distinct moments during a single program execution.

Enabling Multi-Snapshot Mode

To activate multi-snapshotting, invoke magic-trace with the -multi-snapshot true flag. This boolean value propagates through the backend interface defined in src/backend_intf.ml, ultimately altering how the perf command is constructed and how the tracer manages breakpoint lifetimes.

Constructing the Perf Command

In src/perf_tool_backend.ml, the function responsible for building the perf invocation checks the multi_snapshot configuration. When enabled, it appends "--switch-output=signal" to the argument list, instructing perf to listen for a specific signal and write a new snapshot file upon receipt.

The relevant logic appears around lines 502-506:

(* In src/perf_tool_backend.ml – building the perf command *)
let switch_opts =
  match multi_snapshot with
  | true  -> [ "--switch-output=signal" ]   (* <-- enables multi-snapshot *)
  | false -> [] in
let argv = List.concat
  [ [ perf; "record"; "-o"; record_dir ^/ "perf.data"; "--timestamp" ]
  ; event_opts
  ; overwrite_opts
  ; switch_opts                (* this is the key list *)
  ; thread_opts
  ; pid_opt
  ; control_opt
  ; kcore_opts
  ; snapshot_size_opt
  ; Callgraph_mode.to_perf_record_args selected_callgraph_mode
  ] in

With this flag, perf generates a sequence of files such as perf.data, perf.data.1, perf.data.2, and so on, each representing an independent snapshot.

Signal Handling and Forwarding

The implementation relies on SIGUSR2 as the trigger mechanism. Magic-Trace installs a signal handler to forward this signal to the perf child process, coordinating snapshot rotation without stopping the trace.

Handler Setup

The Control module in vendor/tracing/src/tool_output.ml registers the SIGUSR2 handler during initialization. This abstraction provides a clean interface for snapshot management and shutdown procedures.

Triggering Snapshots Programmatically

When a snapshot condition occurs—whether from a user-triggered Ctrl-C or a hit on a specified trigger symbol—the system calls Control.take_snapshot. This function sends SIGUSR2 to the perf child PID:

(* In vendor/tracing/src/tool_output.ml – the Control module *)
let take_snapshot control perf_pid =
  (* Forward the signal that perf expects *)
  Core_unix.kill perf_pid Sys.sigusr2

Perf receives this signal, closes the current output file, and immediately begins writing to the next incremental file in the sequence.

Maintaining Breakpoint Persistence

In multi-snapshot mode, the tracer must keep breakpoints active after the first hit to allow subsequent captures. The logic in src/trace.ml (around lines 389-396) determines whether to disable the breakpoint after a single trigger.

The key variable assignment is:

(* In src/trace.ml – keeping the breakpoint alive for multiple snapshots *)
let single_hit = not opts.multi_snapshot in
if single_hit then Ivar.fill_if_empty done_ivar ()

When multi_snapshot is true, single_hit evaluates to false, preventing the tracer from filling the completion Ivar and terminating the session. This allows the tracing loop to continue, enabling further snapshots via subsequent SIGUSR2 signals.

Summary

• Multi-snapshot mode is activated via the -multi-snapshot true command-line flag.
• The --switch-output=signal argument in src/perf_tool_backend.ml instructs perf to listen for SIGUSR2 to rotate output files.
• The Control module in vendor/tracing/src/tool_output.ml forwards SIGUSR2 to the perf child process to trigger snapshot writing.
• In src/trace.ml, setting single_hit = not opts.multi_snapshot keeps breakpoints active, allowing multiple captures per execution.

Frequently Asked Questions

What is the difference between single and multi-snapshot mode in magic-trace?

Single-snapshot mode captures one trace and exits immediately, while multi-snapshot mode keeps the perf process running, allowing you to capture many snapshots during a single execution by sending multiple SIGUSR2 signals.

Which signal does magic-trace use for switch-output?

Magic-Trace uses SIGUSR2. This specific signal is expected by perf when the --switch-output=signal flag is provided, causing perf to close the current file and begin writing to a new one.

How does magic-trace keep the tracer running between snapshots?

The tracer evaluates let single_hit = not opts.multi_snapshot in src/trace.ml. When multi-snapshot is enabled, this prevents the automatic shutdown logic from triggering after the first breakpoint hit, keeping the trace session alive for additional snapshots.

Where is the signal forwarding logic implemented?

The signal forwarding logic resides in vendor/tracing/src/tool_output.ml within the Control module. This module provides take_snapshot and shutdown functions that manage SIGUSR2 delivery to the child perf process.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}