How to Set Up Observability and Tracing in Cognee: A Complete Guide

Cognee provides a built-in OpenTelemetry-native observability layer that captures spans for pipeline tasks, database queries, LLM calls, and vector searches without requiring external dependencies.

Cognee (topoteretes/cognee) ships with a lightweight, production-ready observability system that makes debugging AI pipelines straightforward. The implementation resides in the cognee.modules.observability package and leverages OpenTelemetry standards to provide end-to-end visibility into your data processing workflows. This guide covers how to enable, configure, and consume observability and tracing in Cognee using both programmatic APIs and environment variables.

Architecture of the Observability System

The observability layer splits responsibilities across three core files in cognee/modules/observability/.

Core Components

tracing.py serves as the core tracing engine. It defines semantic attribute constants (lines 30-46), implements secret redaction (lines 48-66), and provides the CogneeSpanExporter class that buffers the last 50 traces in memory (lines 77-104). This module also contains setup_tracing, shutdown_tracing, and get_tracer helpers (lines 277-332) that manage the OpenTelemetry TracerProvider lifecycle.

trace_context.py acts as the public façade. It exposes functions like enable_tracing, disable_tracing, is_tracing_enabled, get_last_trace, get_all_traces, and clear_traces. These utilities let any part of the codebase toggle tracing at runtime and retrieve buffered spans without importing OpenTelemetry internals directly.

observers.py (optional) defines the Observer abstraction used by modules that emit custom metrics alongside standard traces.

Key Architectural Features

Semantic attribute constants provide a shared vocabulary for all spans. These include COGNEE_PIPELINE_NAME, COGNEE_DB_QUERY, COGNEE_LLM_MODEL, COGNEE_LLM_PROVIDER, and COGNEE_VECTOR_COLLECTION.

Secret redaction automatically scrubs API keys and passwords from span attributes before export, ensuring sensitive credentials never leak into trace data.

In-memory buffering via CogneeSpanExporter stores traces in a thread-safe dictionary, enabling instant retrieval without requiring a running collector.

Enabling Tracing in Cognee

You can activate tracing through three methods depending on your deployment environment.

Programmatic Activation

Import and call enable_tracing from the observability module. Setting console_output=True prints spans to stdout during development:

from cognee.modules.observability import enable_tracing

# Enable tracing with console output for debugging

enable_tracing(console_output=True)

This invokes setup_tracing and sets the internal _tracing_enabled flag (see trace_context.py lines 16-24).

Environment Variable Configuration

Set COGNEE_TRACING_ENABLED=true before starting your application. The is_tracing_enabled function (lines 34-62) reads the base configuration, falls back to this environment variable, and lazily initializes tracing the first time a component checks the flag.

Exporting to External OTLP Backends

To send spans to external collectors like Grafana Tempo, Datadog, or Jaeger, define the OTEL_EXPORTER_OTLP_ENDPOINT environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.dash0.com/api/v1/spans"
export COGNEE_TRACING_ENABLED=true

The _try_add_otlp_exporter helper (lines 252-274 in tracing.py) automatically wires the OTLP exporter to the provider when this variable is present. No code changes are required.

Recording Custom Spans

When instrumenting your own pipeline logic, use the new_span context manager from tracing.py. This creates an OpenTelemetry span, attaches semantic attributes, and applies secret redaction automatically:

from cognee.modules.observability import new_span, COGNEE_PIPELINE_NAME

with new_span("ingestion_task", {COGNEE_PIPELINE_NAME: "document_processor"}):
    # Your business logic here

    process_documents()

The span captures timing, attributes, and any exceptions that occur within the context block.

Retrieving and Analyzing Traces

The in-memory exporter allows immediate access to trace data without querying a remote backend:

from cognee.modules.observability import get_last_trace, get_all_traces, clear_traces

# Retrieve the most recent completed trace

last = get_last_trace()
if last:
    print("Root span:", last.root_span_name)
    for span in last.spans:
        print(f"{span.name}: {span.attributes}")

# Access all buffered traces

all_traces = get_all_traces()
print(f"Collected {len(all_traces)} traces")

# Clear buffer between test runs

clear_traces()

get_last_trace and get_all_traces return CogneeTrace instances that wrap the raw OpenTelemetry ReadableSpan objects (see trace_context.py lines 65-82).

Disabling Tracing

To gracefully shut down the tracing system and flush remaining spans:

from cognee.modules.observability import disable_tracing

disable_tracing()

This calls shutdown_tracing (lines 50-59 in trace_context.py), forces a provider flush, and clears global references to prevent memory leaks in long-running processes.

Integration Points in the Codebase

Cognee's high-level modules already integrate the observability layer, automatically contributing spans when tracing is enabled:

• Vector search operations in cognee/modules/retrieval/utils/node_edge_vector_search.py use new_span with the COGNEE_VECTOR_COLLECTION attribute to track queries against vector databases.
• LLM adapters in cognee/infrastructure/llm/ wrap external API calls, attaching COGNEE_LLM_MODEL and COGNEE_LLM_PROVIDER attributes to identify which model processed each request.
• Database adapters like cognee/infrastructure/databases/graph/neo4j_driver/adapter.py record queries using the COGNEE_DB_QUERY semantic attribute, capturing execution time and query parameters.

These integrations demonstrate the plug-and-play nature of the system—once you call enable_tracing, every component automatically emits detailed telemetry.

Summary

• Cognee's observability layer is OpenTelemetry-native and resides in cognee/modules/observability/.
• Enable tracing programmatically with enable_tracing() or via the COGNEE_TRACING_ENABLED environment variable.
• Export to backends by setting OTEL_EXPORTER_OTLP_ENDPOINT for OTLP-compatible collectors.
• Record spans using the new_span context manager with semantic constants like COGNEE_PIPELINE_NAME.
• Retrieve traces in-memory using get_last_trace() and get_all_traces() without external dependencies.
• Built-in integrations automatically instrument vector searches, LLM calls, and database queries.

Frequently Asked Questions

How do I check if tracing is currently enabled in Cognee?

Call is_tracing_enabled() from cognee.modules.observability. This function checks the internal flag and environment variables, returning a boolean indicating whether the tracing system is active. It also handles lazy initialization if tracing was configured via environment variables but not yet instantiated.

Can I use Cognee tracing without an external collector like Jaeger or Datadog?

Yes. The CogneeSpanExporter buffers the last 50 traces in memory by default. You can retrieve these using get_last_trace() or get_all_traces() without configuring any external endpoints. This is ideal for local development or testing scenarios where you need visibility without infrastructure overhead.

What sensitive data does Cognee automatically redact from traces?

The tracing system scrubs API keys, passwords, and other secrets from span attributes before export (implemented in tracing.py lines 48-66). This redaction applies to all spans created via new_span, ensuring that credentials used for LLM providers or databases do not leak into trace logs or external collectors.

How do I trace custom pipeline steps in my Cognee application?

Import new_span and the relevant semantic constants from cognee.modules.observability. Wrap your logic in a with new_span("step_name", {COGNEE_PIPELINE_NAME: "my_pipeline"}): block. The context manager automatically handles span creation, timing, attribute attachment, and exception recording while applying secret redaction.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}