# Jido Telemetry and Tracing Features: Complete Observability for Elixir Agents

> Unlock complete observability for Elixir agents with Jido's robust telemetry and tracing features. Gain Prometheus metrics, configurable logs, and OpenTelemetry integration.

- Repository: [agentjido/jido](https://github.com/agentjido/jido)
- Tags: deep-dive
- Published: 2026-03-09

---

**Jido ships a fully-featured observability stack that combines structured telemetry events with distributed tracing support, including Prometheus-compatible metrics, configurable log levels, and pluggable tracer backends for seamless OpenTelemetry integration.**

The `agentjido/jido` repository provides comprehensive telemetry and tracing features designed specifically for agent-based Elixir applications. This observability stack enables developers to monitor agent lifecycles, track signal flows across distributed systems, and export metrics to Prometheus while maintaining correlation IDs for end-to-end request tracing.

## Structured Telemetry and Metrics

Jido's telemetry layer, centered in [`lib/jido/telemetry.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/telemetry.ex), provides structured, filterable logging and Prometheus-compatible metrics through three distinct log levels.

### Event Taxonomy and Log Levels

The system distinguishes three effective levels controlled by `Jido.Observe.Config.telemetry_log_level/1`:

- **INFO** – User-visible actions and high-level state changes
- **DEBUG** – Interesting internal operations without excessive noise
- **TRACE** – Full internal churn including every signal processing step

Event names follow a consistent atom list pattern such as `[:jido, :agent, :cmd, :start]` or `[:jido, :agent_server, :signal, :stop]`, enabling precise subscription and filtering.

### Interest Filtering and Smart Logging

At DEBUG level, Jido applies intelligent filtering via `Jido.Telemetry.interesting_signal?/4` and `Jido.Telemetry.interesting_directive?/4`. A signal is logged only when it satisfies at least one predicate:

- Duration exceeds `slow_signal_threshold_ms`
- The signal produced one or more directives
- The signal type is listed in `interesting_signal_types`
- An error occurred during processing

This prevents log flooding while ensuring critical paths remain visible.

### Prometheus-Compatible Metrics

`Jido.Telemetry.metrics/0` builds a set of `Telemetry.Metrics` counters and summaries tracking command counts, signal durations, queue overflow events, and agent lifecycle transitions. These metrics integrate directly with `TelemetryMetricsPrometheus` for scraping by Prometheus or compatible systems.

## Distributed Tracing and Correlation

Jido's tracing infrastructure provides correlation-aware observability across process boundaries and distributed nodes.

### Trace Context Propagation

Every signal can carry a trace payload via the `Jido.Signal.Ext.Trace` extension, containing `trace_id`, `span_id`, `parent_span_id`, and `causation_id`. The helpers in `Jido.Tracing.Trace` create root traces with `new_root/0` and child traces with `child_of/2`, maintaining hierarchical relationships across agent boundaries.

### Process-Local Storage with Jido.Tracing.Context

`Jido.Tracing.Context` stores the current trace in the process dictionary under `{:jido, :trace_context}`, exposing helpers to set, clear, and retrieve the active trace. When processing signals, `ensure_from_signal/1` guarantees a trace is present and stores it for downstream work, while `propagate_to/2` attaches the trace context to child signals.

### Pluggable Tracer Interface

The `Jido.Observe.Tracer` behaviour defines callbacks for `span_start/2`, `span_stop/2`, `span_exception/4`, and optional `with_span_scope/3`. Implementations can forward spans to OpenTelemetry, Datadog, or custom backends. The default `Jido.Observe.NoopTracer` ensures zero overhead when tracing is disabled.

## Configuration and Setup

Observability settings live under the `:observability` key in your application configuration, with legacy support for the `:telemetry` key.

```elixir
config :jido, :observability,
  log_level: :debug,
  tracer: MyApp.JidoTracer,
  tracer_failure_mode: :warn,
  debug_events: :all,
  redact_sensitive: true,
  slow_signal_threshold_ms: 10,
  interesting_signal_types: ["jido.agent.user_request", "jido.llm.done"]

```

Per-instance overrides are supported via the `:jido_instance` metadata key when running multiple Jido instances within a single umbrella or OTP application. Tracer failures are controlled by `tracer_failure_mode`: `:warn` logs the error and continues, while `:strict` raises the exception for development visibility.

## Code Examples

### Synchronous Span with Automatic Tracing

```elixir
Jido.Observe.with_span([:jido, :ai, :tool, :invoke],
  %{tool: "search", query: "Elixir tracing"},
  fn ->
    perform_search()
  end)

```

This emits `[:jido, :ai, :tool, :invoke, :start]` followed by `[:jido, :ai, :tool, :invoke, :stop]` or `[:jido, :ai, :tool, :invoke, :exception]`. The span metadata is automatically enriched with `jido_trace_id`, `jido_span_id`, and related correlation fields, while the configured tracer receives the appropriate callbacks.

### Asynchronous Work and Task Spans

```elixir
span_ctx = Jido.Observe.start_span([:jido, :ai, :llm, :request], %{model: "claude"})

Task.start(fn ->
  try do
    result = call_llm_api()
    Jido.Observe.finish_span(span_ctx, %{output_bytes: byte_size(result)})
    result
  rescue
    e -> Jido.Observe.finish_span_error(span_ctx, :error, e, __STACKTRACE__)
  end
end)

```

The span is created in the parent process using `Jido.Observe.SpanCtx` to track start time and metadata. For cross-process propagation, use `Jido.Tracing.Context.propagate_to/2` before spawning the task, then `ensure_from_signal/1` or manual context restoration in the child process.

### Adding Trace Context to Signals

```elixir

# In a signal handler

{signal, trace} = Jido.Tracing.Context.ensure_from_signal(signal)

# The signal now carries trace metadata for downstream agents

```

When producing child signals:

```elixir
{:ok, child_signal} = Jido.Tracing.Context.propagate_to(new_signal, signal.id)

```

The child inherits the same `trace_id` with a fresh `span_id`, while `parent_span_id` points to the original span. This maintains causal relationships across agent boundaries without external dependencies.

### Custom OpenTelemetry Tracer Implementation

```elixir
defmodule MyApp.JidoTracer do
  @behaviour Jido.Observe.Tracer

  @impl true
  def span_start(event_prefix, metadata) do
    :otel_tracer.start_span(event_prefix, metadata)
  end

  @impl true
  def span_stop(tracer_ctx, measurements) do
    :otel_tracer.end_span(tracer_ctx, measurements)
    :ok
  end

  @impl true
  def span_exception(tracer_ctx, kind, reason, stacktrace) do
    :otel_tracer.record_exception(tracer_ctx, kind, reason, stacktrace)
    :ok
  end
end

```

Configure in [`config/config.exs`](https://github.com/agentjido/jido/blob/main/config/config.exs):

```elixir
config :jido, :observability,
  tracer: MyApp.JidoTracer,
  tracer_failure_mode: :warn

```

Now every Jido span automatically forwards to your OpenTelemetry collector, enabling visualization in Jaeger, Zipkin, or cloud-native observability platforms.

## Key Source Files

| File | Purpose |
|------|---------|
| [`lib/jido/telemetry.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/telemetry.ex) | Core telemetry event handling, log level logic, and metrics definitions. |
| [`lib/jido/telemetry/config.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/telemetry/config.ex) | Backward-compatible shim delegating to `Jido.Observe.Config`. |
| [`lib/jido/telemetry/formatter.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/telemetry/formatter.ex) | Human-readable formatting of log messages. |
| [`lib/jido/observe.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/observe.ex) | High-level façade exposing `with_span/3`, async span API, and enriched telemetry emission. |
| [`lib/jido/observe/config.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/observe/config.ex) | Central configuration for log levels, tracer selection, thresholds, and redaction. |
| [`lib/jido/observe/tracer.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/observe/tracer.ex) | Behaviour definition for tracer backends. |
| [`lib/jido/observe/noop_tracer.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/observe/noop_tracer.ex) | Default no-op tracer ensuring zero overhead when disabled. |
| [`lib/jido/tracing/context.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/tracing/context.ex) | Process-local storage of current trace and propagation helpers. |
| [`lib/jido/tracing/trace.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/tracing/trace.ex) | Creation of root and child trace data and attachment to signals. |
| [`lib/jido/observe/log.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/observe/log.ex) | Thin wrapper around `Logger` respecting configured observability levels. |
| [`lib/jido/observe/span_ctx.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/observe/span_ctx.ex) | Struct holding span start time, metadata, and tracer context for async spans. |

## Summary

- **Jido telemetry and tracing features** provide a two-layer observability stack combining structured logging with distributed tracing capabilities.
- The telemetry layer in [`lib/jido/telemetry.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/telemetry.ex) supports three log levels (INFO, DEBUG, TRACE) with intelligent interest filtering via `interesting_signal?/4` and Prometheus-compatible metrics.
- Distributed tracing is implemented through `Jido.Tracing.Context` for process-local storage and `Jido.Observe.Tracer` behaviour for pluggable backends, enabling correlation across agent boundaries.
- Configuration is centralized in `Jido.Observe.Config` with per-instance override support and failure mode controls (`:warn` vs `:strict`).
- Zero-overhead defaults are ensured by `Jido.Observe.NoopTracer`, while custom implementations can forward spans to OpenTelemetry, Datadog, or proprietary systems.

## Frequently Asked Questions

### How do I configure Jido telemetry and tracing for production use?

Configure the `:observability` key in your [`config/config.exs`](https://github.com/agentjido/jido/blob/main/config/config.exs) with appropriate log levels and tracer modules. For production, set `log_level: :info` or `:warn` to reduce noise, configure `tracer: YourApp.Tracer` to forward to OpenTelemetry, and set `tracer_failure_mode: :warn` to prevent tracing errors from crashing your application. Per-instance overrides via `:jido_instance` metadata allow different configurations for multiple Jido instances within the same OTP application.

### What is the difference between Jido telemetry events and tracing spans?

Telemetry events in [`lib/jido/telemetry.ex`](https://github.com/agentjido/jido/blob/main/lib/jido/telemetry.ex) provide structured logging and metrics emission using the `:telemetry` library, tracking occurrences like `[:jido, :agent, :cmd, :start]`. Tracing spans managed by `Jido.Observe` capture timing and causal relationships across process boundaries, storing trace context in `Jido.Tracing.Context` and propagating correlation IDs via signal extensions. While telemetry answers "what happened," tracing answers "how long did it take and what caused it."

### How does Jido handle trace context across asynchronous processes?

Jido provides `Jido.Tracing.Context` to store the current trace in the process dictionary under `{:jido, :trace_context}`. For async work, use `Jido.Observe.start_span/2` to create a span context struct, then manually propagate the trace to the child process using `Jido.Tracing.Context.propagate_to/2`. In the child process, call `Jido.Tracing.Context.ensure_from_signal/1` or manually restore the context before calling `Jido.Observe.finish_span/2` to complete the distributed trace.

### Can I use Jido telemetry and tracing features without external dependencies?

Yes, Jido works out-of-the-box with zero external tracing dependencies. The default `Jido.Observe.NoopTracer` implementation ensures no overhead when tracing is disabled, while the telemetry system uses only the standard `:telemetry` library included with Elixir. You can emit structured logs and metrics to the console or standard Logger without configuring external collectors. External integrations like OpenTelemetry or Datadog are optional and activated only by implementing the `Jido.Observe.Tracer` behaviour and configuring the tracer module.