# How to Set Up Observability and Tracing in Cognee: A Complete Guide

> Set up observability and tracing in Cognee with this complete guide. Discover how Cognee's OpenTelemetry layer tracks pipeline tasks, DB queries, LLM calls, and vector searches effortlessly.

- Repository: [Topoteretes/cognee](https://github.com/topoteretes/cognee)
- Tags: how-to-guide
- Published: 2026-03-16

---

**Cognee provides a built-in OpenTelemetry-native observability layer that captures spans for pipeline tasks, database queries, LLM calls, and vector searches without requiring external dependencies.**

Cognee (topoteretes/cognee) ships with a lightweight, production-ready observability system that makes debugging AI pipelines straightforward. The implementation resides in the `cognee.modules.observability` package and leverages OpenTelemetry standards to provide end-to-end visibility into your data processing workflows. This guide covers how to enable, configure, and consume observability and tracing in Cognee using both programmatic APIs and environment variables.

## Architecture of the Observability System

The observability layer splits responsibilities across three core files in `cognee/modules/observability/`.

### Core Components

**[`tracing.py`](https://github.com/topoteretes/cognee/blob/main/tracing.py)** serves as the core tracing engine. It defines semantic attribute constants (lines 30-46), implements secret redaction (lines 48-66), and provides the `CogneeSpanExporter` class that buffers the last 50 traces in memory (lines 77-104). This module also contains `setup_tracing`, `shutdown_tracing`, and `get_tracer` helpers (lines 277-332) that manage the OpenTelemetry `TracerProvider` lifecycle.

**[`trace_context.py`](https://github.com/topoteretes/cognee/blob/main/trace_context.py)** acts as the public façade. It exposes functions like `enable_tracing`, `disable_tracing`, `is_tracing_enabled`, `get_last_trace`, `get_all_traces`, and `clear_traces`. These utilities let any part of the codebase toggle tracing at runtime and retrieve buffered spans without importing OpenTelemetry internals directly.

**[`observers.py`](https://github.com/topoteretes/cognee/blob/main/observers.py)** (optional) defines the `Observer` abstraction used by modules that emit custom metrics alongside standard traces.

### Key Architectural Features

**Semantic attribute constants** provide a shared vocabulary for all spans. These include `COGNEE_PIPELINE_NAME`, `COGNEE_DB_QUERY`, `COGNEE_LLM_MODEL`, `COGNEE_LLM_PROVIDER`, and `COGNEE_VECTOR_COLLECTION`.

**Secret redaction** automatically scrubs API keys and passwords from span attributes before export, ensuring sensitive credentials never leak into trace data.

**In-memory buffering** via `CogneeSpanExporter` stores traces in a thread-safe dictionary, enabling instant retrieval without requiring a running collector.

## Enabling Tracing in Cognee

You can activate tracing through three methods depending on your deployment environment.

### Programmatic Activation

Import and call `enable_tracing` from the observability module. Setting `console_output=True` prints spans to stdout during development:

```python
from cognee.modules.observability import enable_tracing

# Enable tracing with console output for debugging

enable_tracing(console_output=True)

```

This invokes `setup_tracing` and sets the internal `_tracing_enabled` flag (see [`trace_context.py`](https://github.com/topoteretes/cognee/blob/main/trace_context.py) lines 16-24).

### Environment Variable Configuration

Set `COGNEE_TRACING_ENABLED=true` before starting your application. The `is_tracing_enabled` function (lines 34-62) reads the base configuration, falls back to this environment variable, and lazily initializes tracing the first time a component checks the flag.

### Exporting to External OTLP Backends

To send spans to external collectors like **Grafana Tempo**, **Datadog**, or **Jaeger**, define the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable:

```bash
export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.dash0.com/api/v1/spans"
export COGNEE_TRACING_ENABLED=true

```

The `_try_add_otlp_exporter` helper (lines 252-274 in [`tracing.py`](https://github.com/topoteretes/cognee/blob/main/tracing.py)) automatically wires the OTLP exporter to the provider when this variable is present. No code changes are required.

## Recording Custom Spans

When instrumenting your own pipeline logic, use the `new_span` context manager from [`tracing.py`](https://github.com/topoteretes/cognee/blob/main/tracing.py). This creates an OpenTelemetry span, attaches semantic attributes, and applies secret redaction automatically:

```python
from cognee.modules.observability import new_span, COGNEE_PIPELINE_NAME

with new_span("ingestion_task", {COGNEE_PIPELINE_NAME: "document_processor"}):
    # Your business logic here

    process_documents()

```

The span captures timing, attributes, and any exceptions that occur within the context block.

## Retrieving and Analyzing Traces

The in-memory exporter allows immediate access to trace data without querying a remote backend:

```python
from cognee.modules.observability import get_last_trace, get_all_traces, clear_traces

# Retrieve the most recent completed trace

last = get_last_trace()
if last:
    print("Root span:", last.root_span_name)
    for span in last.spans:
        print(f"{span.name}: {span.attributes}")

# Access all buffered traces

all_traces = get_all_traces()
print(f"Collected {len(all_traces)} traces")

# Clear buffer between test runs

clear_traces()

```

`get_last_trace` and `get_all_traces` return `CogneeTrace` instances that wrap the raw OpenTelemetry `ReadableSpan` objects (see [`trace_context.py`](https://github.com/topoteretes/cognee/blob/main/trace_context.py) lines 65-82).

## Disabling Tracing

To gracefully shut down the tracing system and flush remaining spans:

```python
from cognee.modules.observability import disable_tracing

disable_tracing()

```

This calls `shutdown_tracing` (lines 50-59 in [`trace_context.py`](https://github.com/topoteretes/cognee/blob/main/trace_context.py)), forces a provider flush, and clears global references to prevent memory leaks in long-running processes.

## Integration Points in the Codebase

Cognee's high-level modules already integrate the observability layer, automatically contributing spans when tracing is enabled:

- **Vector search operations** in [`cognee/modules/retrieval/utils/node_edge_vector_search.py`](https://github.com/topoteretes/cognee/blob/main/cognee/modules/retrieval/utils/node_edge_vector_search.py) use `new_span` with the `COGNEE_VECTOR_COLLECTION` attribute to track queries against vector databases.
- **LLM adapters** in `cognee/infrastructure/llm/` wrap external API calls, attaching `COGNEE_LLM_MODEL` and `COGNEE_LLM_PROVIDER` attributes to identify which model processed each request.
- **Database adapters** like [`cognee/infrastructure/databases/graph/neo4j_driver/adapter.py`](https://github.com/topoteretes/cognee/blob/main/cognee/infrastructure/databases/graph/neo4j_driver/adapter.py) record queries using the `COGNEE_DB_QUERY` semantic attribute, capturing execution time and query parameters.

These integrations demonstrate the plug-and-play nature of the system—once you call `enable_tracing`, every component automatically emits detailed telemetry.

## Summary

- **Cognee's observability layer** is OpenTelemetry-native and resides in `cognee/modules/observability/`.
- **Enable tracing** programmatically with `enable_tracing()` or via the `COGNEE_TRACING_ENABLED` environment variable.
- **Export to backends** by setting `OTEL_EXPORTER_OTLP_ENDPOINT` for OTLP-compatible collectors.
- **Record spans** using the `new_span` context manager with semantic constants like `COGNEE_PIPELINE_NAME`.
- **Retrieve traces** in-memory using `get_last_trace()` and `get_all_traces()` without external dependencies.
- **Built-in integrations** automatically instrument vector searches, LLM calls, and database queries.

## Frequently Asked Questions

### How do I check if tracing is currently enabled in Cognee?

Call `is_tracing_enabled()` from `cognee.modules.observability`. This function checks the internal flag and environment variables, returning a boolean indicating whether the tracing system is active. It also handles lazy initialization if tracing was configured via environment variables but not yet instantiated.

### Can I use Cognee tracing without an external collector like Jaeger or Datadog?

Yes. The `CogneeSpanExporter` buffers the last 50 traces in memory by default. You can retrieve these using `get_last_trace()` or `get_all_traces()` without configuring any external endpoints. This is ideal for local development or testing scenarios where you need visibility without infrastructure overhead.

### What sensitive data does Cognee automatically redact from traces?

The tracing system scrubs API keys, passwords, and other secrets from span attributes before export (implemented in [`tracing.py`](https://github.com/topoteretes/cognee/blob/main/tracing.py) lines 48-66). This redaction applies to all spans created via `new_span`, ensuring that credentials used for LLM providers or databases do not leak into trace logs or external collectors.

### How do I trace custom pipeline steps in my Cognee application?

Import `new_span` and the relevant semantic constants from `cognee.modules.observability`. Wrap your logic in a `with new_span("step_name", {COGNEE_PIPELINE_NAME: "my_pipeline"}):` block. The context manager automatically handles span creation, timing, attribute attachment, and exception recording while applying secret redaction.