# Tracing and Observability Features in aisuite: A Complete Technical Guide

> Explore aisuite's comprehensive tracing and observability features. Understand every LLM run step, model, and tool interaction with a searchable web UI. Your technical guide awaits.

- Repository: [Andrew Ng/aisuite](https://github.com/andrewyng/aisuite)
- Tags: deep-dive
- Published: 2026-07-27

---

**aisuite implements a full-stack tracing and observability system that records every step of an LLM run—from the high-level run lifecycle down to individual model and tool interactions—and renders the data in a searchable, filterable web UI.**

This guide explores the tracing and observability features available in the [andrewyng/aisuite](https://github.com/andrewyng/aisuite) repository, covering how the framework captures execution events, stores them persistently, and provides real-time visibility through a built-in viewer. The system is designed to work out-of-the-box while remaining modular enough to integrate with external observability platforms.

## Core Architecture of the Tracing System

The observability stack in aisuite consists of four loosely-coupled layers: **trace events**, **trace sinks**, **trace stores**, and a **viewer server**. Each layer is implemented as a distinct Python module, allowing you to substitute components as needed.

### Trace Events: Immutable Execution Records

At the foundation of the system lies the `TraceEvent` dataclass defined in [`aisuite/tracing/sinks.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/sinks.py). These are immutable records describing a single point in time, such as `run.started`, `model.send`, or `tool.completed`.

Every event carries:
- A `trace_id` (UUID) for correlation
- Timestamps for precise timing
- Optional span hierarchy for nested operations
- A free-form `data` payload containing request parameters, responses, and metadata

### Trace Sinks: Pluggable Event Back-ends

Sinks are responsible for emitting events to their destinations. The framework ships with three built-in implementations in [`aisuite/tracing/sinks.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/sinks.py):

- **LocalTraceSink**: Writes events as JSON-Lines to a local file (default: `.aisuite/events.jsonl`)
- **HttpTraceSink**: POSTs events to a remote HTTP endpoint for external collection
- **InMemoryTraceSink**: Retains events in RAM, ideal for unit testing

The `TraceSink` protocol allows you to define custom sinks by implementing the `emit(event: TraceEvent)` method.

### Trace Stores: Persistent Storage and Reconstruction

Stores handle the append, list, and reconstruction operations. Located in [`aisuite/tracing/store.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/store.py), two concrete implementations are provided:

- **JsonlTraceStore**: File-backed storage using JSON-Lines format
- **InMemoryTraceStore**: Volatile storage for testing and ephemeral runs

The `reconstruct_runs()` function merges raw events by `trace_id`, infers run status (`running`, `completed`, or `failed`), and calculates derived metrics including duration, message counts, and tool event tallies.

### Viewer Server: Real-time Web Interface

The `ViewerServer` class in [`aisuite/tracing/viewer.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/viewer.py) hosts a lightweight HTTP server that serves a single-page React-style UI. It exposes endpoints like `/api/runs` that return pre-processed summaries generated by `prepare_viewer_run_summaries()` and detailed views from `prepare_viewer_runs()`.

The browser client polls the API every 1.5 seconds to display:
- Timeline visualizations of event sequences
- Tool argument and result previews
- Approval/denial decision logs
- Latency statistics and artifact references

## How Tracing Works in Practice

Understanding the data flow helps you optimize observability for production workloads.

1. **Context Creation**: When an agent starts, [`aisuite/agents/context.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/agents/context.py) generates a UUID `trace_id` and attaches the list of active `trace_sinks` to the runtime context. This context propagates to all tools and sub-agents automatically.

2. **Event Emission**: Framework components call `emit_event(sinks, event)` whenever significant actions occur. For example, sending an LLM request creates a `model.send` event, while the response generates `model.response`. Tool interactions emit `tool.allowed`, `tool.started`, and `tool.completed` events.

3. **Storage**: Each sink writes to its downstream store. The default CLI configuration uses a `LocalTraceSink` writing to `.aisuite/events.jsonl`, while you can optionally add an `HttpTraceSink` to forward copies to external collectors like OpenTelemetry.

4. **Run Reconstruction**: When viewing history, `JsonlTraceStore.list_runs()` reads the raw JSONL, and `reconstruct_runs()` aggregates events by `trace_id` to produce normalized run dictionaries with computed fields.

5. **Live Visualization**: Calling `start_viewer()` spawns a server that reads the trace store, builds UI-ready structures, and serves the bundled static assets from `viewer-ui/dist` (or the repository directory during development).

## Configuring Tracing and Observability in aisuite

The public `ai.tracing` namespace exposes all observability primitives. Here is how to use the key features in your applications.

### Emitting Custom Trace Events

Create and emit custom events to track application-specific logic:

```python
from aisuite.tracing import TraceEvent, emit_event, get_configured_sinks

# Create a custom event

event = TraceEvent(
    event_type="custom.validation",
    trace_id="trace_123",
    agent_name="my_agent",
    run_name="demo_run",
)

# Emit to all configured sinks

emit_event(get_configured_sinks(), event)

```

### Starting the Local Viewer

Launch the web UI to inspect runs in real-time:

```python
from aisuite.tracing import start_viewer

viewer = start_viewer(trace_file="my_trace.jsonl", port=8765)
print("Open your browser at:", viewer.url)  # http://127.0.0.1:8765

```

The viewer automatically refreshes every 1.5 seconds and displays timeline events, tool panels, and latency charts without external dependencies.

### Reading Trace Files Programmatically

Process historical traces for testing or reporting:

```python
from aisuite.tracing import read_trace_file

runs = read_trace_file("my_trace.jsonl")
print("Found runs:", len(runs))
print("First run status:", runs[0]["status"])
print("Duration:", runs[0].get("duration_ms"), "ms")

```

### Forwarding Events to External Collectors

Configure an HTTP sink to integrate with enterprise observability platforms:

```python
from aisuite.tracing import HttpTraceSink, configure

http_sink = HttpTraceSink(
    "http://otel-collector.example.com/v1/traces",
    fail_silently=False
)

configure(http_sink)  # All subsequent emit_event() calls forward to the endpoint

```

## Summary

- **aisuite/tracing/sinks.py** defines `TraceEvent` and three built-in sinks (`LocalTraceSink`, `HttpTraceSink`, `InMemoryTraceSink`) with the `emit_event()` helper.
- **aisuite/tracing/store.py** implements `JsonlTraceStore` and `InMemoryTraceStore`, plus the `reconstruct_runs()` logic that aggregates raw events into queryable run summaries.
- **aisuite/tracing/viewer.py** hosts the `ViewerServer` and preparation functions that transform stored events into the web UI format.
- The [`aisuite/agents/context.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/agents/context.py) module automatically provisions `trace_id` and `trace_sinks` to all agent executions, ensuring zero-instrumentation observability.
- Use `start_viewer()` to launch the local UI, `read_trace_file()` for programmatic access, and `configure()` with `HttpTraceSink` for external forwarding.

## Frequently Asked Questions

### How do I enable tracing in an existing aisuite application?

Tracing is enabled by default when using the standard CLI or agent constructors. The framework automatically creates a `LocalTraceSink` writing to `.aisuite/events.jsonl`. You can customize sinks by calling `configure()` with specific sink instances before starting your agents, or by setting the `trace_sinks` parameter in the agent context.

### What is the difference between a trace sink and a trace store?

A **trace sink** is an active emitter that receives events as they happen and forwards them to destinations (files, HTTP endpoints, or memory). A **trace store** is a persistent storage abstraction that supports appending, listing, and reconstructing historical runs. The `LocalTraceSink` uses a `JsonlTraceStore` internally, while `HttpTraceSink` sends data to remote collectors without local persistence.

### Can I use aisuite tracing with OpenTelemetry or Datadog?

Yes. Implement the `TraceSink` protocol to create a custom sink that formats `TraceEvent` objects into OpenTelemetry spans or Datadog logs, then pass it to `configure()`. Alternatively, use `HttpTraceSink` to POST events to a proxy service that translates aisuite's JSON format into your observability platform's native protocol.

### Where are the web UI static assets located?

The viewer serves static assets from the `viewer-ui/dist` directory within the repository, or from bundled package data when installed via pip. The `ViewerServer` in [`aisuite/tracing/viewer.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/viewer.py) handles routing, API endpoints, and static file serving automatically when you call `start_viewer()`.