# How Tracing Works in aisuite Agent Execution: Events, Stores, and Viewer

> Understand aisuite agent execution tracing. Learn how events, stores, and the viewer visualize LLM runs, tool use, and errors in a searchable timeline.

- Repository: [Andrew Ng/aisuite](https://github.com/andrewyng/aisuite)
- Tags: internals
- Published: 2026-07-30

---

**aisuite captures every step of an LLM-agent run as structured trace events, persists them via configurable sinks to JSONL files or memory, and provides a built-in web viewer to visualize model calls, tool executions, and errors in a searchable timeline.**

Tracing in aisuite agent execution provides complete observability into how large language model agents interact with tools and models. The `andrewyng/aisuite` repository implements a three-stage pipeline—event generation, persistent storage, and interactive visualization—that records every significant operation from initial prompt to final output.

## The Three-Component Tracing Architecture

The tracing system in aisuite consists of three integrated layers that handle event capture, storage, and display.

### Trace Sinks ([`aisuite/tracing/sinks.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/sinks.py))

**Trace sinks** act as the collectors. They receive event dictionaries from the running agent and forward them to a persistence layer. The module provides three primary implementations:

- **`InMemoryTraceSink`** – Buffers events in Python memory for unit testing or temporary inspection.
- **`LocalTraceSink`** – Writes events to a local JSONL file via `JsonlTraceStore`.
- **`HttpTraceSink`** – POSTs events to a remote HTTP endpoint for centralized logging.

Each sink implements the `emit(record)` method, which receives a dictionary containing the event type and payload.

### Trace Stores ([`aisuite/tracing/store.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/store.py))

**Trace stores** handle durability and retrieval. The two concrete implementations are:

- **`InMemoryTraceStore`** – Maintains records in a `list[dict]` with methods like `list_runs()` and `get_run(trace_id)`.
- **`JsonlTraceStore`** – Appends each event as a single JSON line to a `.jsonl` file, providing read-only queries without loading the entire file into memory.

The store abstraction allows the viewer and analysis tools to query run metadata and event histories uniformly, regardless of the underlying storage medium.

### The Viewer Server ([`aisuite/tracing/viewer.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/viewer.py))

The **`ViewerServer`** serves a minimal web interface that renders the stored runs. It exposes REST endpoints (`/api/runs`, `/api/runs/<trace_id>`) and static assets from `viewer-ui/static/viewer`. When you call `start_viewer()`, the server periodically refreshes the trace file (every 1.5 seconds) to show live updates during long-running agent executions.

## Event Generation During Agent Execution

During execution, the agent emits discrete events that mark state transitions. Each event is a dictionary with an **`event_type`** string and a **`data`** payload containing context-specific details.

Critical event types include:

- **Lifecycle**: `run.started`, `run.completed`, `run.failed`
- **Model interaction**: `model.send`, `model.response`, `model.error`
- **Tool governance**: `tool.allowed`, `tool.denied`
- **Tool execution**: `tool.started`, `tool.completed`, `tool.failed`

The helpers in [`aisuite/tracing/normalize.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/normalize.py) (e.g., `_event_summary`, `_event_tone`) transform these raw events into human-readable text for the UI, extracting tool names, argument previews, and error summaries.

## Persisting Events with Sinks and Stores

The connection between sinks and stores is implemented in `TraceStoreSink`, found in [`aisuite/tracing/sinks.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/sinks.py):

```python
class TraceStoreSink(TraceSink):
    def __init__(self, store: TraceStore):
        self.store = store

    def emit(self, record: dict[str, Any]) -> None:
        self.store.append_records([record])

```

When using **`JsonlTraceStore`**, each `emit` call appends a line to the file. The public API in [`aisuite/tracing/__init__.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/__init__.py) provides `read_trace_file()` to load these events:

```python
def read_trace_file(trace_file: str | Path) -> list[dict[str, Any]]:
    return prepare_viewer_runs(JsonlTraceStore(trace_file).list_runs())

```

This function returns UI-ready structures without requiring manual JSON parsing.

## Transforming Raw Events into Display Summaries

Before rendering, raw events undergo transformation via **`prepare_viewer_runs`** (called by the viewer and `read_trace_file`). This pipeline:

1. **Collects hierarchy metadata** – Maps `parent_run_id` relationships and calculates `child_count`.
2. **Extracts aggregates** – Gathers tool names, approval statuses, error counts, and token usage statistics.
3. **Generates display dictionaries** – Creates a `display` object containing:
   - `title` and `subtitle` for the run header
   - `status` tone (success, error, warning)
   - `timeline` preview items

The **`_sanitize_for_viewer`** function ensures long strings are truncated to compact previews while preserving artifact references for downstream debugging.

## Aggregating Activities for the Timeline

The viewer groups granular events into higher-level **activities** using **`_run_activities`** in [`aisuite/tracing/viewer.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/viewer.py). This aggregation logic identifies:

- **Model calls** – Pairs of `model.send` and `model.response` events merged into a single activity with `duration_ms`.
- **Tool calls** – Sequences of `tool.allowed`/`denied`, `tool.started`, and `tool.completed`/`failed` collapsed into one item showing approval status and execution result.
- **Standalone events** – Unpaired events like `run.failed` displayed individually.

Each activity object contains:
- `duration_ms`, `tone`, `title`, `summary`
- Optional **`tool_name`**, **`model`**, **`approval`** status
- **Artifact references** pointing to full input/output data

These activities populate the `timeline` array that drives the visual debugger.

## Starting the Trace Viewer

To inspect traces interactively, use **`start_viewer`** from [`aisuite/tracing/__init__.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/__init__.py):

```python
import ai.tracing as tracing
from pathlib import Path

viewer = tracing.start_viewer(trace_file=Path("run.jsonl"), port=0)
print(f"Open {viewer.url} to inspect the trace")

```

Setting `port=0` allows the OS to assign a free port. The server serves:
- **Static UI** – HTML/CSS from `viewer-ui/static/viewer` or the packaged `PACKAGE_UI_DIST`.
- **REST API** – `/api/runs` lists all runs; `/api/runs/<trace_id>` returns full run details including the `display` object and raw events.

## Practical Code Examples

### Enable Tracing to a JSONL File

```python
import ai
import ai.tracing as tracing
from pathlib import Path

trace_file = Path("agent_run.jsonl")
trace_sink = tracing.LocalTraceSink(trace_file)

client = ai.Client(trace_sink=trace_sink)
result = client.run(prompt="Generate a Python function to sort a list")

```

### Launch the Local Web Viewer

```python
import ai.tracing as tracing
from pathlib import Path

viewer = tracing.start_viewer(trace_file=Path("agent_run.jsonl"), port=0)

# Opens browser to http://127.0.0.1:8000/?embed=1

```

### Query Trace Data Programmatically

```python
import ai.tracing as tracing
from pathlib import Path

runs = tracing.read_trace_file(Path("agent_run.jsonl"))
for run in runs:
    print(f"{run['trace_id']}: {run['status']} - {run['display']['title']}")

```

### Access Detailed Run Information

```python
state = tracing.ViewerTraceState(trace_file=Path("agent_run.jsonl"))
detail = state.get_run("trace_abc123")
print(detail["final_output"])
print(detail["display"]["timeline"][0])  # First activity

```

## Summary

- **aisuite/agent execution tracing** relies on three layers: sinks ([`aisuite/tracing/sinks.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/sinks.py)) for collection, stores ([`aisuite/tracing/store.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/store.py)) for persistence, and the viewer ([`aisuite/tracing/viewer.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/tracing/viewer.py)) for visualization.
- Events capture every model call, tool approval, and execution step with structured payloads.
- **`LocalTraceSink`** combined with **`JsonlTraceStore`** provides durable, queryable logs without external dependencies.
- The **`start_viewer`** function launches a self-contained web server that aggregates raw events into a navigable timeline via `prepare_viewer_runs` and `_run_activities`.

## Frequently Asked Questions

### What event types does aisuite trace during agent execution?

aisuite emits lifecycle events (`run.started`, `run.completed`, `run.failed`), model interaction events (`model.send`, `model.response`, `model.error`), and comprehensive tool events (`tool.allowed`, `tool.denied`, `tool.started`, `tool.completed`, `tool.failed`). Each event contains an `event_type` and a `data` payload with arguments, results, or error details, as defined in the tracing normalization logic.

### How do I configure tracing to write to a local JSONL file?

Instantiate **`tracing.LocalTraceSink`** with a `Path` object pointing to your desired `.jsonl` file, then pass that sink to the `ai.Client` constructor via the `trace_sink` parameter. The `LocalTraceSink` automatically uses `JsonlTraceStore` to append each event as a new JSON line, making the file safe for concurrent writes and easy to parse later.

### Can I view traces without using the built-in web server?

Yes. Use **`tracing.read_trace_file()`** to load a JSONL trace into a list of run dictionaries, or instantiate **`tracing.ViewerTraceState`** directly to query specific runs. These APIs provide the same data structures used by the web viewer—including the `display` summary and `timeline` activities—allowing you to analyze agent behavior in Jupyter notebooks or custom dashboards.

### What is the performance overhead of enabling tracing in aisuite?

Overhead is minimal for the **`JsonlTraceStore`** because events are appended incrementally to disk without locking, and the default sink buffers in memory before writing. **`InMemoryTraceSink`** adds no I/O overhead but consumes RAM proportional to the number of events. For high-throughput production environments, consider using **`HttpTraceSink`** to offload storage to a remote service, keeping the agent process unblocked.