Jido Telemetry and Tracing Features: Complete Observability for Elixir Agents

Jido ships a fully-featured observability stack that combines structured telemetry events with distributed tracing support, including Prometheus-compatible metrics, configurable log levels, and pluggable tracer backends for seamless OpenTelemetry integration.

The agentjido/jido repository provides comprehensive telemetry and tracing features designed specifically for agent-based Elixir applications. This observability stack enables developers to monitor agent lifecycles, track signal flows across distributed systems, and export metrics to Prometheus while maintaining correlation IDs for end-to-end request tracing.

Structured Telemetry and Metrics

Jido's telemetry layer, centered in lib/jido/telemetry.ex, provides structured, filterable logging and Prometheus-compatible metrics through three distinct log levels.

Event Taxonomy and Log Levels

The system distinguishes three effective levels controlled by Jido.Observe.Config.telemetry_log_level/1:

• INFO – User-visible actions and high-level state changes
• DEBUG – Interesting internal operations without excessive noise
• TRACE – Full internal churn including every signal processing step

Event names follow a consistent atom list pattern such as [:jido, :agent, :cmd, :start] or [:jido, :agent_server, :signal, :stop], enabling precise subscription and filtering.

Interest Filtering and Smart Logging

At DEBUG level, Jido applies intelligent filtering via Jido.Telemetry.interesting_signal?/4 and Jido.Telemetry.interesting_directive?/4. A signal is logged only when it satisfies at least one predicate:

• Duration exceeds slow_signal_threshold_ms
• The signal produced one or more directives
• The signal type is listed in interesting_signal_types
• An error occurred during processing

This prevents log flooding while ensuring critical paths remain visible.

Prometheus-Compatible Metrics

Jido.Telemetry.metrics/0 builds a set of Telemetry.Metrics counters and summaries tracking command counts, signal durations, queue overflow events, and agent lifecycle transitions. These metrics integrate directly with TelemetryMetricsPrometheus for scraping by Prometheus or compatible systems.

Distributed Tracing and Correlation

Jido's tracing infrastructure provides correlation-aware observability across process boundaries and distributed nodes.

Trace Context Propagation

Every signal can carry a trace payload via the Jido.Signal.Ext.Trace extension, containing trace_id, span_id, parent_span_id, and causation_id. The helpers in Jido.Tracing.Trace create root traces with new_root/0 and child traces with child_of/2, maintaining hierarchical relationships across agent boundaries.

Process-Local Storage with Jido.Tracing.Context

Jido.Tracing.Context stores the current trace in the process dictionary under {:jido, :trace_context}, exposing helpers to set, clear, and retrieve the active trace. When processing signals, ensure_from_signal/1 guarantees a trace is present and stores it for downstream work, while propagate_to/2 attaches the trace context to child signals.

Pluggable Tracer Interface

The Jido.Observe.Tracer behaviour defines callbacks for span_start/2, span_stop/2, span_exception/4, and optional with_span_scope/3. Implementations can forward spans to OpenTelemetry, Datadog, or custom backends. The default Jido.Observe.NoopTracer ensures zero overhead when tracing is disabled.

Configuration and Setup

Observability settings live under the :observability key in your application configuration, with legacy support for the :telemetry key.

config :jido, :observability,
  log_level: :debug,
  tracer: MyApp.JidoTracer,
  tracer_failure_mode: :warn,
  debug_events: :all,
  redact_sensitive: true,
  slow_signal_threshold_ms: 10,
  interesting_signal_types: ["jido.agent.user_request", "jido.llm.done"]

Per-instance overrides are supported via the :jido_instance metadata key when running multiple Jido instances within a single umbrella or OTP application. Tracer failures are controlled by tracer_failure_mode: :warn logs the error and continues, while :strict raises the exception for development visibility.

Code Examples

Synchronous Span with Automatic Tracing

Jido.Observe.with_span([:jido, :ai, :tool, :invoke],
  %{tool: "search", query: "Elixir tracing"},
  fn ->
    perform_search()
  end)

This emits [:jido, :ai, :tool, :invoke, :start] followed by [:jido, :ai, :tool, :invoke, :stop] or [:jido, :ai, :tool, :invoke, :exception]. The span metadata is automatically enriched with jido_trace_id, jido_span_id, and related correlation fields, while the configured tracer receives the appropriate callbacks.

Asynchronous Work and Task Spans

span_ctx = Jido.Observe.start_span([:jido, :ai, :llm, :request], %{model: "claude"})

Task.start(fn ->
  try do
    result = call_llm_api()
    Jido.Observe.finish_span(span_ctx, %{output_bytes: byte_size(result)})
    result
  rescue
    e -> Jido.Observe.finish_span_error(span_ctx, :error, e, __STACKTRACE__)
  end
end)

The span is created in the parent process using Jido.Observe.SpanCtx to track start time and metadata. For cross-process propagation, use Jido.Tracing.Context.propagate_to/2 before spawning the task, then ensure_from_signal/1 or manual context restoration in the child process.

Adding Trace Context to Signals

# In a signal handler

{signal, trace} = Jido.Tracing.Context.ensure_from_signal(signal)

# The signal now carries trace metadata for downstream agents

When producing child signals:

{:ok, child_signal} = Jido.Tracing.Context.propagate_to(new_signal, signal.id)

The child inherits the same trace_id with a fresh span_id, while parent_span_id points to the original span. This maintains causal relationships across agent boundaries without external dependencies.

Custom OpenTelemetry Tracer Implementation

defmodule MyApp.JidoTracer do
  @behaviour Jido.Observe.Tracer

  @impl true
  def span_start(event_prefix, metadata) do
    :otel_tracer.start_span(event_prefix, metadata)
  end

  @impl true
  def span_stop(tracer_ctx, measurements) do
    :otel_tracer.end_span(tracer_ctx, measurements)
    :ok
  end

  @impl true
  def span_exception(tracer_ctx, kind, reason, stacktrace) do
    :otel_tracer.record_exception(tracer_ctx, kind, reason, stacktrace)
    :ok
  end
end

Configure in config/config.exs:

config :jido, :observability,
  tracer: MyApp.JidoTracer,
  tracer_failure_mode: :warn

Now every Jido span automatically forwards to your OpenTelemetry collector, enabling visualization in Jaeger, Zipkin, or cloud-native observability platforms.

Key Source Files

File	Purpose
lib/jido/telemetry.ex	Core telemetry event handling, log level logic, and metrics definitions.
lib/jido/telemetry/config.ex	Backward-compatible shim delegating to Jido.Observe.Config.
lib/jido/telemetry/formatter.ex	Human-readable formatting of log messages.
lib/jido/observe.ex	High-level façade exposing with_span/3, async span API, and enriched telemetry emission.
lib/jido/observe/config.ex	Central configuration for log levels, tracer selection, thresholds, and redaction.
lib/jido/observe/tracer.ex	Behaviour definition for tracer backends.
lib/jido/observe/noop_tracer.ex	Default no-op tracer ensuring zero overhead when disabled.
lib/jido/tracing/context.ex	Process-local storage of current trace and propagation helpers.
lib/jido/tracing/trace.ex	Creation of root and child trace data and attachment to signals.
lib/jido/observe/log.ex	Thin wrapper around Logger respecting configured observability levels.
lib/jido/observe/span_ctx.ex	Struct holding span start time, metadata, and tracer context for async spans.

Summary

• Jido telemetry and tracing features provide a two-layer observability stack combining structured logging with distributed tracing capabilities.
• The telemetry layer in lib/jido/telemetry.ex supports three log levels (INFO, DEBUG, TRACE) with intelligent interest filtering via interesting_signal?/4 and Prometheus-compatible metrics.
• Distributed tracing is implemented through Jido.Tracing.Context for process-local storage and Jido.Observe.Tracer behaviour for pluggable backends, enabling correlation across agent boundaries.
• Configuration is centralized in Jido.Observe.Config with per-instance override support and failure mode controls (:warn vs :strict).
• Zero-overhead defaults are ensured by Jido.Observe.NoopTracer, while custom implementations can forward spans to OpenTelemetry, Datadog, or proprietary systems.

Frequently Asked Questions

How do I configure Jido telemetry and tracing for production use?

Configure the :observability key in your config/config.exs with appropriate log levels and tracer modules. For production, set log_level: :info or :warn to reduce noise, configure tracer: YourApp.Tracer to forward to OpenTelemetry, and set tracer_failure_mode: :warn to prevent tracing errors from crashing your application. Per-instance overrides via :jido_instance metadata allow different configurations for multiple Jido instances within the same OTP application.

What is the difference between Jido telemetry events and tracing spans?

Telemetry events in lib/jido/telemetry.ex provide structured logging and metrics emission using the :telemetry library, tracking occurrences like [:jido, :agent, :cmd, :start]. Tracing spans managed by Jido.Observe capture timing and causal relationships across process boundaries, storing trace context in Jido.Tracing.Context and propagating correlation IDs via signal extensions. While telemetry answers "what happened," tracing answers "how long did it take and what caused it."

How does Jido handle trace context across asynchronous processes?

Jido provides Jido.Tracing.Context to store the current trace in the process dictionary under {:jido, :trace_context}. For async work, use Jido.Observe.start_span/2 to create a span context struct, then manually propagate the trace to the child process using Jido.Tracing.Context.propagate_to/2. In the child process, call Jido.Tracing.Context.ensure_from_signal/1 or manually restore the context before calling Jido.Observe.finish_span/2 to complete the distributed trace.

Can I use Jido telemetry and tracing features without external dependencies?

Yes, Jido works out-of-the-box with zero external tracing dependencies. The default Jido.Observe.NoopTracer implementation ensures no overhead when tracing is disabled, while the telemetry system uses only the standard :telemetry library included with Elixir. You can emit structured logs and metrics to the console or standard Logger without configuring external collectors. External integrations like OpenTelemetry or Datadog are optional and activated only by implementing the Jido.Observe.Tracer behaviour and configuring the tracer module.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}