How to Trace Python Code Execution with pytest: Using the cgr Call-Graph Plugin

The cgr pytest plugin automatically records detailed runtime call-graphs for every test, attributing each function call to the specific test that invoked it via sys.monitoring (PEP 669).

The vitali87/code-graph-rag repository provides a pytest plugin that makes it straightforward to trace Python code execution with pytest without modifying test code. By leveraging Python 3.12's sys.monitoring interface, the plugin captures precise caller-callee relationships and maps them to specific test cases. When enabled via command-line flag, it produces structured trace files showing exactly which functions each test exercised.

How the pytest Plugin Captures Execution Traces

The plugin implementation in codebase_rag/trace/pytest_plugin.py orchestrates the tracing lifecycle, while the core instrumentation logic resides in codebase_rag/trace/tracer.py. The architecture follows four distinct phases:

Instrumentation via sys.monitoring

During pytest configuration, the plugin checks for the --cgr-trace flag. When present, it registers a profiler tool using Python's sys.monitoring API (PEP 669) and initializes the CallGraphTracer for the repository root. This occurs in pytest_configure within the plugin file, ensuring the tracer is active before any tests execute.

Test Isolation with Workload Identifiers

To attribute calls to specific tests, the plugin hooks into pytest_runtest_setup and pytest_runtest_teardown. Before each test runs, it extracts the test's node ID (e.g., tests/test_mod.py::test_func) and sets it as the current workload in the tracer. After the test completes, the workload is cleared. This mechanism ensures every recorded call pair in tracer.py carries accurate provenance indicating which test triggered the execution.

Filtering Repository-Local Calls

The CallGraphTracer defined in codebase_rag/trace/tracer.py applies strict filtering logic to reduce noise. It collects call-pair statistics—caller function, callee function, call count, and optional receiver type—only for code residing under the specified repository root. Third-party packages, standard library modules, and generated files are automatically excluded from the trace, keeping the output focused on your codebase.

JSON-Lines Output Format

When the pytest session finishes, the plugin triggers trace serialization via pytest_sessionfinish. The tracer writes aggregated data to a JSON-Lines file (default path defined in codebase_rag/constants.py). The format consists of a header record followed by individual call records, structured according to the schema defined in codebase_rag/trace/records.py.

Running pytest with Code Tracing Enabled

The plugin remains completely inactive unless explicitly invoked, ensuring zero overhead during standard test runs.

Basic Tracing

To capture execution traces for a single test file:

pytest tests/my_module_test.py --cgr-trace

This generates a trace file containing the header and one line per observed caller-callee pair for all tests in the file.

Custom Output Paths and Repository Roots

For monorepos or specific output requirements, specify custom paths:

pytest -m "not integration" \
       --cgr-trace \
       --cgr-trace-output ./traces/run1.trace.jsonl \
       --cgr-trace-repo /path/to/my/repo

• --cgr-trace-output sets the destination file path.
• --cgr-trace-repo restricts tracing to files under the given root, excluding external dependencies.

Parallel Execution with pytest-xdist

The plugin supports distributed testing via pytest-xdist. Each worker process writes its own trace file with a distinct suffix:

pytest -n 4 --cgr-trace --cgr-trace-output ./traces/parallel.trace.jsonl

After execution, merge the worker-specific files:

cat ./traces/parallel.trace.jsonl-* > ./traces/parallel.trace.jsonl

The order of concatenation does not affect analysis since each record carries independent workload identifiers.

Reading Generated Trace Files

Process the JSON-Lines output programmatically using the records utility:

from pathlib import Path
from codebase_rag.trace.records import read_trace_file

header, records = read_trace_file(Path("run1.trace.jsonl"))
print(f"Trace generated by: {header.tracer}")

for rec in records:
    print(f"{rec.caller.qualname} → {rec.callee.qualname} ({rec.count} calls)")

This extracts the header metadata and yields structured records containing qualified function names, call counts, and associated test workloads.

Summary

• The cgr pytest plugin in codebase_rag/trace/pytest_plugin.py enables zero-config execution tracing via the --cgr-trace flag.
• It uses sys.monitoring (PEP 669) to capture call graphs with minimal runtime overhead.
• Each test receives a unique workload identifier (node ID) that tags all calls made during its execution.
• The CallGraphTracer filters calls to include only repository-local code, ignoring third-party libraries.
• Output uses a JSON-Lines format defined in codebase_rag/trace/records.py, supporting both single-process and parallel (pytest-xdist) workflows.

Frequently Asked Questions

Does tracing slow down my test suite?

When the --cgr-trace flag is omitted, the plugin is completely inert and adds zero overhead. When enabled, the overhead depends on call frequency, but the sys.monitoring implementation in codebase_rag/trace/tracer.py is optimized to record only essential call-pair data rather than full stack traces, keeping performance impact minimal for most test suites.

Can I use this with pytest-xdist for parallel testing?

Yes. The plugin explicitly handles parallel execution by assigning unique trace file suffixes to each xdist worker (e.g., -gw0, -gw1). After the run, concatenate the worker files using standard shell tools to produce a unified trace for analysis.

What file format does the trace output use?

The tracer writes JSON-Lines (.jsonl) files as specified in codebase_rag/trace/records.py. Each file begins with a header object containing metadata (tracer version, timestamp), followed by one JSON object per line representing individual call records with fields for caller, callee, count, and workload ID.

How do I limit tracing to specific directories?

Use the --cgr-trace-repo argument to specify the repository root directory. The CallGraphTracer compares every code object against this root path, excluding any calls originating from or targeting files outside this boundary. This is particularly useful in monorepos where you want to trace only specific sub-packages.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}