Magic-Trace Collection Mode Auto-Detection and Fallback Behavior

Magic-trace automatically selects Intel Processor Trace (PT) when available by probing /sys/bus/event_source/devices/intel_pt, falls back to stacktrace sampling with a warning if PT is absent, and allows explicit override via the -sampling flag.

Magic-trace, Jane Street's high-performance tracing tool, intelligently adapts to your hardware capabilities through automatic collection mode detection. The tool's selection logic, implemented in OCaml within src/collection_mode.ml, determines whether to use hardware-assisted Intel PT or software-based sampling without requiring manual configuration.

The Two Collection Modes

Magic-trace supports two distinct tracing backends:

• Intel Processor Trace (PT): Leverages hardware features to record precise instruction-level traces with minimal overhead. This mode requires specific CPU support and kernel exposure via the debug filesystem.
• Stacktrace Sampling: A lightweight software-based approach that periodically captures call stacks. This mode serves as the universal fallback when PT hardware is unavailable or when explicitly requested by the user.

Both modes support additional performance events through the -events flag, allowing you to collect metrics like cache-misses or branch-misses alongside the core trace data.

How Auto-Detection Works in src/collection_mode.ml

The auto-detection mechanism relies on a simple filesystem probe. Before launching the tracer, magic-trace checks for the existence of the Intel PT device node at /sys/bus/event_source/devices/intel_pt.

This check occurs through Core_unix.access with the `Exists flag:

match Core_unix.access "/sys/bus/event_source/devices/intel_pt" [ `Exists ] with
| Ok ()    -> Intel_processor_trace { extra_events }
| Error _  ->
  Core.eprintf "Intel PT support not found. magic-trace will continue and use sampling instead.\n";
  Stacktrace_sampling { extra_events }

If the path exists and is accessible, the tool assumes PT hardware and kernel drivers are present. If the probe fails, magic-trace gracefully degrades to sampling mode after printing a diagnostic warning to stderr.

Understanding the select_collection_mode Logic

The core decision function select_collection_mode in src/collection_mode.ml implements a priority-based selection strategy:

let select_collection_mode ~extra_events ~use_sampling =
  match use_sampling with
  | true  -> Stacktrace_sampling { extra_events }
  | false ->
    (match Core_unix.access "/sys/bus/event_source/devices/intel_pt" [ `Exists ] with
     | Ok ()    -> Intel_processor_trace { extra_events }
     | Error _  ->
       Core.eprintf
         "Intel PT support not found. magic-trace will continue and use sampling instead.\n";
       Stacktrace_sampling { extra_events })

The logic follows this exact precedence:

1. Explicit user override: If -sampling is passed (use_sampling = true), the function immediately returns Stacktrace_sampling, bypassing hardware detection entirely.
2. Hardware probe: Without the override, the function checks for the PT device node.
3. Fallback activation: When the probe fails, the function emits the warning message and returns the sampling mode constructor.

User Overrides and Extra Events

Magic-trace provides two command-line flags that influence collection mode selection:

• -sampling: A boolean flag that forces stacktrace sampling mode regardless of PT availability. Use this when you prefer the sampling approach or need to avoid PT-specific limitations.
• -events: Accepts a comma-separated list of additional events (e.g., cache-misses, branch-misses) that augment the trace data. These events work in both PT and sampling modes.

The extra_events parameter in select_collection_mode carries these user-specified events into whichever mode is ultimately selected.

Integration with the Magic-Trace Pipeline

The collection mode selection integrates into the broader architecture through two additional files:

• src/subcommand.ml: Connects the command-line flag parsing to select_collection_mode, passing the parsed use_sampling boolean and extra_events list.
• src/trace.ml: Consumes the returned variant (either Intel_processor_trace or Stacktrace_sampling) to instantiate the appropriate backend writer. This separation ensures the collection mode decision happens early, while the actual tracing implementation remains mode-agnostic until instantiation.

Practical Usage Examples

Run magic-trace with default auto-detection:

# Auto-detect: uses PT if available, otherwise sampling with warning

magic-trace run ./my_binary

Force sampling mode explicitly:

# Bypass PT detection even on capable hardware

magic-trace -sampling run ./my_binary

Add hardware events to either mode:

# Works with both PT and sampling modes

magic-trace -events cache-misses,branch-misses run ./my_binary

Summary

• Magic-trace collection mode auto-detection probes /sys/bus/event_source/devices/intel_pt to determine Intel PT availability.
• The select_collection_mode function in src/collection_mode.ml implements a three-tier logic: explicit override first, hardware probe second, fallback warning third.
• Stacktrace sampling serves as the universal fallback when PT hardware is absent or when the user passes -sampling.
• Additional events via -events are supported in both collection modes.
• The selection propagates through src/subcommand.ml and materializes in src/trace.ml through distinct backend instantiation.

Frequently Asked Questions

How does magic-trace detect Intel PT support?

Magic-trace checks for the existence of /sys/bus/event_source/devices/intel_pt using Core_unix.access with the `Exists flag according to the implementation in src/collection_mode.ml. If this sysfs path is present, the tool assumes the CPU and kernel support Intel Processor Trace and selects PT mode automatically.

What happens if I run magic-trace on a machine without Intel PT?

When the filesystem probe fails, magic-trace prints "Intel PT support not found. magic-trace will continue and use sampling instead." to stderr via Core.eprintf and automatically falls back to Stacktrace_sampling mode. The tool continues execution without requiring manual intervention.

Can I force magic-trace to use sampling even if PT is available?

Yes. Pass the -sampling flag on the command line. When use_sampling is set to true in select_collection_mode, the function immediately selects Stacktrace_sampling without performing the hardware probe, effectively bypassing PT detection on capable systems.

Are extra events like cache-misses available in both modes?

Yes. The -events flag accepts a comma-separated list of events that are passed through the extra_events parameter to both Intel_processor_trace and Stacktrace_sampling constructors. These performance counter events work regardless of which collection mode is active.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}