# How to Ensure Explainability in AI Agents: A Trajectory-Centric Architecture

> Ensure AI agent explainability by recording interactions as immutable trajectories for deterministic verification, policy checking, and visual replay. Learn the trajectory-centric architecture.

- Repository: [Bojie Li/ai-agent-book](https://github.com/bojieli/ai-agent-book)
- Tags: architecture
- Published: 2026-07-30

---

**AI agent explainability is guaranteed by recording every interaction as an immutable trajectory that supports deterministic verification, policy checking, and visual replay.**

Ensuring explainability in AI agents requires capturing the complete reasoning chain—from initial thought to final action—in a structured, auditable format. The `bojieli/ai-agent-book` repository demonstrates this through a trajectory-centric architecture that logs every LLM message, tool call, and observation as an immutable sequence. This design enables three core mechanisms: full execution tracing, multi-layer verification, and interactive visual replay.

## Implement Trajectory-Centric Logging

The foundation of explainability lies in the `self.trajectory` list initialized in [`self_evolving_tools/agent.py`](https://github.com/bojieli/ai-agent-book/blob/main/self_evolving_tools/agent.py). This chronological log captures every observable event as the agent executes, creating an immutable record that can be replayed or audited.

### Structure the Execution Trace

Each step appended to the trajectory contains a typed payload that categorizes the agent's internal state:

- **`thought`** – The reasoning or planning phase before action selection
- **`action`** – Tool invocations with parameters and metadata
- **`observation`** – Raw outputs returned from tool executions
- **`answer`** – The final synthesized response delivered to the user

In [`chapter8/self-evolving-tools/agent.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter8/self-evolving-tools/agent.py), the agent populates this log during the execution loop:

```python
class Agent:
    def __init__(self):
        self.trajectory = []                     # ← Initialise empty log

    def run(self, task):
        # 1️⃣ Thought

        self.trajectory.append({"type": "thought", "content": self.think(task)})
        # 2️⃣ Action (tool call)

        result = self.call_tool(task)
        self.trajectory.append({"type": "action", "tool": "search", "result": result})
        # 3️⃣ Observation

        self.trajectory.append({"type": "observation", "content": result["output"]})
        # 4️⃣ Answer

        answer = self.synthesize(result)
        self.trajectory.append({"type": "answer", "content": answer})
        return answer

```

This structured format ensures that every decision point is preserved with full context, making the agent's logic transparent without requiring code inspection.

## Apply Three-Layer Verification

Before a trajectory is accepted as successful, it passes through a deterministic verification pipeline implemented in [`chapter8/trajectory-verifier/verifier.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter8/trajectory-verifier/verifier.py). This system produces structured `DimensionResult` objects that explain precisely why a trajectory passed or failed.

### ResultVerifier for Final State Validation

The **ResultVerifier** compares the final environment state against expected outcomes, checking for mismatched state keys or incorrect values. This layer ensures the agent actually achieved the task objective rather than merely terminating.

### ProcessVerifier for Policy Compliance

The **ProcessVerifier** enforces operational constraints including privacy rules, grounding requirements, and promise-action consistency. This layer detects policy violations such as leaked sensitive values or hallucinated tool parameters that violate defined schemas.

### HeuristicQualityJudge for Deterministic Scoring

The **HeuristicQualityJudge** provides an evidence-citing quality score that mimics LLM-based rubrics without non-deterministic inference. It cites specific trajectory segments to justify its scoring, producing auditable metrics rather than opaque ratings.

Execute verification using the `TrajectoryVerifier` class:

```python
from chapter8.trajectory_verifier.verifier import TrajectoryVerifier

verifier = TrajectoryVerifier()
report = verifier.evaluate(trajectory)   # `trajectory` is the dict recorded above

print(report["overall_score"], report["dimension_results"])

```

## Visualize Agent Reasoning with Interactive Replay

Raw JSON trajectories become human-readable through the `agent-trajectory` web component in [`extras/agent-lab/agent-trajectory.js`](https://github.com/bojieli/ai-agent-book/blob/main/extras/agent-lab/agent-trajectory.js). This UI renders the execution flow as an interactive timeline with color-coded badges for each step type.

Embed the component in documentation or debugging interfaces:

```html
<agent-trajectory src="/extras/agent-lab/data/ch1-asean-capitals-gpt5.json"></agent-trajectory>

```

The component automatically fetches the trajectory JSON (defined by the schema in [`extras/agent-lab/SCHEMA.md`](https://github.com/bojieli/ai-agent-book/blob/main/extras/agent-lab/SCHEMA.md)), parses each step, and displays the reasoning flow visually. This bridges the gap between machine-structured logs and human understanding, allowing developers and end-users to trace exactly how the agent reached its conclusion.

## Enable Cross-Trajectory Analysis

Advanced explainability requires analyzing patterns across multiple executions. In [`chapter8/self-modifying-agent/evolution.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter8/self-modifying-agent/evolution.py), the system detects cross-trajectory patterns before triggering any self-modification, ensuring that code changes are justified by historical evidence rather than single-instance anomalies.

Similarly, the tool-reuse detection logic in [`self-evolving-tools/agent.py`](https://github.com/bojieli/ai-agent-book/blob/main/self-evolving-tools/agent.py) records whether a tool succeeded in previous trajectories, allowing the agent to cite specific historical evidence when answering queries. This creates a feedback loop where the agent's evolving behavior remains traceable to concrete prior executions.

## Summary

- **Immutable trajectory logs** capture every thought, action, observation, and answer as structured JSON, enabling complete audit trails in `self.trajectory`.
- **Three-layer verification** combines ResultVerifier, ProcessVerifier, and HeuristicQualityJudge to produce deterministic, evidence-based quality assessments.
- **Visual replay components** render trajectories as interactive timelines, making LLM reasoning transparent to non-technical stakeholders.
- **Cross-trajectory pattern detection** ensures that self-modifying agents only evolve based on justified, observable historical patterns.

## Frequently Asked Questions

### What data structure should an AI agent trajectory use?

A trajectory should be implemented as a chronological list of dictionaries, where each entry contains a `type` field (thought, action, observation, answer) and relevant metadata such as tool names, raw outputs, and timestamps. The `bojieli/ai-agent-book` repository defines this schema formally in [`extras/agent-lab/SCHEMA.md`](https://github.com/bojieli/ai-agent-book/blob/main/extras/agent-lab/SCHEMA.md) to ensure compatibility with verification tools and visual components.

### How does three-layer verification improve AI agent transparency?

The three-layer verification exposes precise evidence for both success and failure modes: ResultVerifier validates final outcomes against ground truth, ProcessVerifier checks for policy violations during execution, and HeuristicQualityJudge provides deterministic scoring with citations to specific trajectory segments. Together they produce `DimensionResult` objects that explain exactly which constraints were satisfied or violated, eliminating opaque pass/fail judgments.

### Can trajectory logs support automated agent self-modification?

Yes. The [`evolution.py`](https://github.com/bojieli/ai-agent-book/blob/main/evolution.py) module demonstrates that agents can inspect prior trajectories to detect recurring patterns before modifying their own code. By requiring cross-trajectory evidence for any self-modification, the system ensures that behavioral changes are justified by historical data rather than spurious single-instance observations, maintaining explainability even as the agent evolves.

### What is the best way to visualize LLM agent reasoning steps?

Embed the [`agent-trajectory.js`](https://github.com/bojieli/ai-agent-book/blob/main/agent-trajectory.js) web component, which reads JSON-encoded trajectories and renders them as interactive timelines with color-coded badges. This approach requires no custom frontend code—simply reference the trajectory file and the component handles parsing, step-type classification, and visual layout automatically.