Magic-Trace Full Execution Mode vs Snapshot Ring Buffer: Key Differences Explained

Full-execution mode records every event continuously until process termination, while the snapshot ring buffer (default) uses a bounded circular buffer that captures periodic slices when triggered, drastically reducing memory overhead.

Magic-Trace, Jane Street's performance tracing tool, offers two distinct data collection strategies for the perf backend that trade completeness against resource efficiency. Understanding the difference between magic-trace full execution mode and the snapshot ring buffer is essential for effective profiling, as choosing the wrong approach can result in multi-gigabyte trace files or missed critical events between capture points.

How Full Execution Mode Works

When you invoke Magic-Trace with the -full-execution flag, the tool disables the snapshot mechanism entirely and opens an unbounded recording stream. In src/perf_tool_backend.ml (lines 28-35), this flag is explicitly defined to "record a program's full execution instead of using a snapshot ring buffer."

This mode creates a linear trace that grows continuously for the entire lifetime of the traced process. According to the source code documentation, the trace size can increase by hundreds of MB per second, and the viewer may struggle to load files larger than approximately 100 MiB. Use this mode only when you require an exhaustive, complete record of every single event from startup to termination, and ensure you have sufficient disk space to accommodate the unbounded output.

How the Snapshot Ring Buffer Works (Default)

By default, Magic-Trace employs a circular kernel buffer that operates as a ring buffer. When this buffer fills, the kernel takes a snapshot—writing the accumulated events to a file and clearing the buffer for the next batch. This cycle repeats throughout the program's execution, preserving only the most recent window of activity.

The buffer size remains strictly bounded: approximately 4 MiB for root users and 256 KiB for non-root users. Because older data is overwritten as new events arrive, memory usage stays constant regardless of how long the program runs, though you lose any events that fall between snapshot triggers.

As implemented in src/trace.ml, the function Backend.Recording.maybe_take_snapshot handles the snapshot logic, while src/when_to_snapshot.ml defines the trigger mechanisms—such as the -trigger <symbol> breakpoint option, SIGINT (Ctrl+C), or process exit—that determine when the buffer contents are flushed to disk.

Technical Implementation Details

The architectural distinction between these modes resides in the Recording.Control type and the create function within src/perf_tool_backend.ml. When the -full-execution flag is absent, the backend selects the appropriate snapshot mechanism—either using a perf control-fd request or a signal such as SIGUSR2—based on capabilities detected in src/perf_capabilities.ml.

If the flag is present, the backend bypasses the snapshot selection logic entirely and instructs perf to keep the event stream open until the process exits. This path eliminates the ring buffer management entirely, resulting in the continuous data accumulation characteristic of full-execution mode.

Practical Usage Examples

Choose your collection mode based on the debugging scenario and expected trace duration:

# Default snapshot mode - suitable for long-running processes

magic-trace run ./my_program

# Snapshot triggered by specific function entry

magic-trace run -trigger my_func ./my_program

# Full execution - captures everything until exit (caution: large files)

magic-trace run -full-execution ./my_program

In the first two examples, Magic-Trace utilizes the ring buffer and only persists data when snapshots trigger. In the third example, the -full-execution flag disables the snapshot machinery in src/perf_tool_backend.ml, causing the backend to record all events without bound.

Summary

• Full-execution mode (-full-execution) creates an unbounded trace of all events from startup to termination, consuming hundreds of MB per second and producing files that may exceed 100 MiB and overwhelm the viewer.
• Snapshot ring buffer (default) uses a circular buffer of 4 MiB (root) or 256 KiB (non-root) that captures discrete execution slices when triggered by signals, breakpoints, or process exit, ensuring constant memory usage.
• The implementation distinction resides in src/perf_tool_backend.ml, specifically in the flag definition (lines 28-35) and the Recording.Control logic that selects between continuous recording and snapshot-based collection.
• Use full-execution only for complete traces of short-lived programs; use the default snapshot mode for long-running services or when you need specific temporal slices around known trigger points.

Frequently Asked Questions

When should I use magic-trace full execution mode instead of the default snapshot ring buffer?

Use full-execution mode when debugging complex interactions that require an exhaustive, unbroken event timeline from start to finish. According to the source code in src/perf_tool_backend.ml, this mode is appropriate only when you can tolerate trace files growing at hundreds of MB per second and potentially exceeding the viewer's 100 MiB practical limit.

How large does the snapshot ring buffer get in magic-trace?

The snapshot ring buffer remains bounded to approximately 4 MiB for root users and 256 KiB for non-root users, regardless of execution duration. This fixed size ensures constant memory usage, though it means events occurring between snapshots are permanently discarded unless explicitly captured by a trigger.

What triggers a snapshot in the ring buffer mode?

Snapshots trigger on process exit, SIGINT (Ctrl+C), or user-specified breakpoints defined via the -trigger <symbol> flag. The logic for these triggers resides in src/when_to_snapshot.ml, while the actual snapshot capture is handled by Backend.Recording.maybe_take_snapshot in src/trace.ml.

Why does the viewer struggle with full-execution traces larger than 100 MiB?

Full-execution traces contain every recorded event for the entire process lifetime without the downsampling inherent to ring buffer snapshots. The Magic-Trace viewer is optimized for the smaller, discrete capture windows produced by snapshot mode, and loading monolithic continuous traces that exceed approximately 100 MiB can cause significant performance degradation or memory exhaustion.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}