How to Set Up and Configure Tracing with Custom Processors in openai-agents-python

You can set up and configure tracing with custom processors in openai-agents-python by subclassing TracingProcessor, implementing its lifecycle hooks, and registering your instance with the global TraceProvider before any traces are created.

The openai-agents-python SDK provides a built-in tracing subsystem that records the lifecycle of agent runs and their operations. To integrate this telemetry with external observability platforms or custom logging systems, you can set up and configure tracing with custom processors that handle trace and span events according to your own logic.

Core Tracing Architecture

The tracing system is built around a provider-processor-exporter chain defined in src/agents/tracing/.

TraceProvider and DefaultTraceProvider

The TraceProvider (src/agents/tracing/provider.py) acts as an abstract factory that creates traces and spans and manages the processor chain. The DefaultTraceProvider is the concrete implementation instantiated on first import. It lazily creates a BatchTraceProcessor and a BackendSpanExporter unless explicitly configured otherwise.

Key methods include:

• register_processor(proc) – appends a processor to the chain.
• set_processors([proc]) – replaces the entire processor list.
• create_trace() / create_span() – factory methods used by the public API.

TracingProcessor Interface

All custom processors must inherit from TracingProcessor (src/agents/tracing/processor_interface.py). This abstract base class defines five lifecycle hooks:

def on_trace_start(self, trace): ...
def on_trace_end(self, trace): ...
def on_span_start(self, span): ...
def on_span_end(self, span): ...

Additionally, implementations should provide shutdown() and force_flush() for clean-up and deterministic export.

Default Implementations

• BatchTraceProcessor (src/agents/tracing/processors.py) – Buffers spans and traces in a thread-safe queue, exporting them in batches via a background worker.
• BackendSpanExporter (src/agents/tracing/processors.py) – Concrete exporter that POSTs JSON payloads to the OpenAI tracing ingest endpoint, handling authentication and retry logic.

Implementing a Custom Tracing Processor

To create a custom processor, subclass TracingProcessor and implement the required hooks. Below is a minimal example that logs lifecycle events to stdout:

from agents.tracing.processor_interface import TracingProcessor

class LoggingProcessor(TracingProcessor):
    """Logs trace and span boundaries to the console."""

    def on_trace_start(self, trace):
        print(f"[TRACE START] {trace.name} ({trace.trace_id})")

    def on_trace_end(self, trace):
        print(f"[TRACE END]   {trace.name} ({trace.trace_id})")

    def on_span_start(self, span):
        print(f"  [SPAN START] {span.span_data.__class__.__name__} ({span.span_id})")

    def on_span_end(self, span):
        print(f"  [SPAN END]   {span.span_data.__class__.__name__} ({span.span_id})")

    def shutdown(self):
        print("[SHUTDOWN] Processor shutting down")

    def force_flush(self):
        print("[FLUSH] Force flush called")

Registering and Configuring Your Processor

Registration must occur before the first trace is created. The TraceProvider is lazily instantiated on the first call to get_trace_provider(), so you must register your processor immediately after importing the SDK.

from agents.tracing import get_trace_provider, trace, agent_span

# 1. Obtain the global provider (creates the singleton if needed)

provider = get_trace_provider()

# 2. Register the custom processor

provider.register_processor(LoggingProcessor())

# 3. Now traces will invoke your processor

with trace("production-workflow") as t:
    t.start()
    with agent_span(name="assistant-1") as span:
        span.start()
        # ... agent logic ...

        span.finish()
    t.finish()

To replace the entire processor chain (removing the default batch exporter), use set_processors():

provider.set_processors([LoggingProcessor()])  # Only your processor will run

Advanced Configuration Options

Disabling Tracing Globally

Set the environment variable OPENAI_AGENTS_DISABLE_TRACING=true before importing the SDK. The DefaultTraceProvider checks this flag during initialization and substitutes no-op implementations.

export OPENAI_AGENTS_DISABLE_TRACING=true
python my_agent.py

Per-Trace API Keys

Pass a TracingConfig instance to the trace() factory to supply a specific API key for that trace, overriding the global default:

from agents.tracing import trace, TracingConfig

config = TracingConfig(api_key="sk-custom-key")
with trace("secure-job", tracing=config) as t:
    t.start()
    # ...

    t.finish()

The key is propagated automatically to child spans via the provider's create_trace method.

Forcing Flush and Shutdown

For deterministic export in short-lived scripts (e.g., serverless functions), call force_flush() before exit:

provider = get_trace_provider()
provider.force_flush()  # Blocks until queued items are exported

Common Pitfalls and Troubleshooting

Symptom	Root Cause	Solution
Custom processor never receives events	Processor registered after the first trace was created (provider already initialized with default chain).	Register the processor immediately after SDK import, before any trace() or agent.run() calls.
No data appears in OpenAI dashboard	Missing OPENAI_API_KEY environment variable and no explicit key in TracingConfig.	Export the key or pass it via TracingConfig(api_key="...").
Traces dropped under heavy load	Default BatchTraceProcessor queue size (8192) exceeded.	Increase max_queue_size when constructing BatchTraceProcessor manually: BatchTraceProcessor(exporter, max_queue_size=20000).
Silent export failures	Exporter logs at warning level; root logger may be filtered.	Set logging.getLogger("agents.tracing").setLevel(logging.DEBUG) to surface retry logic and payload details.

Summary

• The openai-agents-python SDK uses a TraceProvider singleton (src/agents/tracing/provider.py) to manage tracing lifecycle.
• To create a custom processor, inherit from TracingProcessor (src/agents/tracing/processor_interface.py) and implement on_trace_start, on_trace_end, on_span_start, and on_span_end.
• Register your processor early using get_trace_provider().register_processor() before any traces are created to ensure it receives all events.
• Use TracingConfig to supply per-trace API keys and set OPENAI_AGENTS_DISABLE_TRACING=true to disable tracing globally.
• Call force_flush() on the provider before short-lived processes exit to ensure all telemetry is exported.

Frequently Asked Questions

How do I ensure my custom processor receives every trace event?

Register your processor immediately after importing the SDK and before invoking any high-level APIs like agents.run() or trace(). The TraceProvider is lazily initialized on first access, so any traces created before registration will not invoke your processor. Use get_trace_provider().register_processor(MyProcessor()) at module load time.

Can I use multiple processors simultaneously?

Yes. The TraceProvider maintains a list of processors. You can append additional processors with register_processor() or replace the entire chain with set_processors([proc1, proc2]). Each processor's hooks are invoked sequentially in the order registered.

How do I disable the default OpenAI backend exporter while keeping my custom processor?

After obtaining the provider, call set_processors() with a list containing only your custom implementation. This removes the default BatchTraceProcessor and BackendSpanExporter that ship with the SDK:

provider = get_trace_provider()
provider.set_processors([MyCustomProcessor()])

What is the difference between shutdown() and force_flush()?

force_flush() blocks the calling thread until all queued traces and spans are exported, but keeps the processor alive for future events. shutdown() permanently stops the processor, tears down background threads, and releases resources; after shutdown, the processor will not accept new traces. Call force_flush() before short-lived scripts exit, and shutdown() during graceful application termination.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}