How to Trace Running Processes with magic-trace attach: A Complete Guide

TLDR: Run magic-trace attach -pid <PID> to capture Intel Processor Trace data from live Linux processes, optionally using -trigger <symbol> to snapshot on function entry and -multi-snapshot for continuous recording.

magic-trace is Jane Street's open-source performance analysis tool that records Intel PT (Processor Trace) execution flows. The magic-trace attach subcommand enables you to trace already-running processes without restarting them, making it essential for diagnosing production workloads or attaching to long-lived daemons.

How the attach Command Works

The attach subcommand orchestrates three distinct layers: CLI parsing, attachment logic, and backend recording. Understanding this architecture helps explain how the tool safely intercepts running processes.

CLI Parsing and Flag Handling

In src/trace.ml (lines 90-103), the command-line interface is built using Command.async_or_error. This defines the attach subcommand's summary, help text, and available flags including -pid, -trigger, and -multi-snapshot. When invoked, the parser validates inputs and prepares the recording options structure.

Process Selection and ELF Discovery

If you omit the -pid flag, magic-trace invokes select_pid (lines 113-120 in src/trace.ml) to present a fuzzy-finder interface for selecting target processes. Once a PID is identified, the tool reads /proc/<pid>/exe to locate the ELF binary (lines 124-129), which is required for resolving symbol triggers and understanding the executable's layout.

Recording Initialization

Before attachment, the system constructs Record_opts containing timer resolution, trace scope, and trigger configuration. The core tracing machinery initializes via Backend.Recording.attach_and_record (lines 152-160 in src/trace.ml), which sets up the perf infrastructure while the target process continues executing.

Runtime Attachment and Event Loop

The attach function (lines 16-34 in src/trace.ml) implements the critical attachment logic. It creates hardware breakpoints if a trigger symbol was specified, then enters a monitoring loop that waits for either:

• A Ctrl-C signal (SIGINT) from the user
• A breakpoint hit if using trigger-based snapshotting

When either event occurs, Backend.Recording.maybe_take_snapshot dumps the PT ring buffer to a compressed .fxt.gz file. Unless -multi-snapshot is enabled, the command exits after the first snapshot. Finally, cleanup (lines 35-43) detaches from the process, destroys breakpoints, and writes metadata to the record_dir.

Practical Usage Examples

These commands demonstrate common magic-trace attach workflows based on the implementation details above.

Attach to an Interactive Process

Launch the fuzzy finder to select a running process visually:

magic-trace attach

Attach to a Specific PID

For scripted workflows or when you know the target process ID:

magic-trace attach -pid 12345

Trigger-Based Snapshotting

Capture a trace only when a specific function executes. The ? flag opens an interactive symbol selector:

magic-trace attach -pid 12345 -trigger ?

Or specify the symbol directly:

magic-trace attach -pid 12345 -trigger my_function_name

Continuous Multi-Snapshot Mode

Record multiple snapshots without exiting on the first trigger:

magic-trace attach -pid 12345 -trigger my_function -multi-snapshot

Key Source Files Reference

Understanding the implementation requires familiarity with these modules:

• src/trace.ml – Contains the CLI definition in attach_command, PID selection logic in select_pid, and the core attach function that orchestrates breakpoint setup and the monitoring loop.
• src/backend_intf.ml – Defines the abstract interface for recording backends, specifically the attach_and_record signature.
• src/perf_tool_backend.ml – Implements the backend interface by driving the Linux perf tool for PT trace collection.
• src/direct_backend/manual_perf.ml – Provides low-level FFI wrappers around perf system calls, including magic_recording_attach_stub.

Summary

• magic-trace attach traces live processes via Intel PT without requiring process restarts.
• The command supports fuzzy PID selection when -pid is omitted, falling back to an interactive picker implemented in select_pid.
• ELF discovery automatically reads /proc/<pid>/exe to resolve symbols for trigger setup.
• Trigger-based snapshots capture traces when specific functions execute, while -multi-snapshot enables continuous recording through the monitoring loop.
• Output files use the .fxt.gz format, ready for analysis at https://magic-trace.org/.

Frequently Asked Questions

How does magic-trace attach select a process if I don't specify a PID?

According to src/trace.ml (lines 113-120), the tool invokes select_pid to launch a fuzzy-finder interface that lists running processes, allowing you to select targets interactively rather than providing numeric process IDs.

What file format does magic-trace attach generate?

The tool produces compressed .fxt.gz files via Backend.Recording.maybe_take_snapshot. These traces contain Intel PT data and are viewable in the Magic Trace web UI at https://magic-trace.org/.

Can I attach to a process without stopping or modifying it?

Yes. As implemented in src/trace.ml (lines 16-34), the attachment uses non-invasive perf infrastructure that records the process while it runs normally. You only need read access to /proc/<pid>/exe and the ability to use perf events.

How do I capture multiple snapshots from a single attach session?

Use the -multi-snapshot flag. By default, magic-trace attach exits after the first snapshot (triggered by Ctrl-C or breakpoint hit), but with -multi-snapshot, the monitoring loop continues collecting traces via maybe_take_snapshot until you manually terminate the command.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}