How Dexter's Agent Loop Manages Tool Execution and Iteration: A Deep Dive into the Core Architecture

Dexter's agent loop is an event-driven async generator that repeatedly calls the LLM, executes detected tool calls via AgentToolExecutor, appends full results to a Scratchpad, and iterates until the model returns a final answer or the maximum iteration limit is reached.

The virattt/dexter repository implements a deterministic agent loop that orchestrates multi-step reasoning by binding tools to the LLM, parsing tool calls from responses, and managing context through an append-only scratchpad. This architecture enables complex financial analysis workflows where the agent must iteratively retrieve data, perform calculations, and synthesize answers.

Core Architecture of Dexter's Agent Loop

Initialization and RunContext Setup

The agent loop begins in src/agent/agent.ts with the Agent.create() static method. This factory function initializes the system prompt, fetches available tools via getTools(), and constructs a RunContext object that maintains mutable state throughout the execution.

// src/agent/agent.ts
const agent = Agent.create({ maxIterations: 8 });

The createRunContext() function in src/agent/run-context.ts instantiates a fresh Scratchpad—an append-only JSONL log that records every tool result without summarization. This design choice ensures complete observability while enabling Anthropic-style context management.

The Main Iteration Cycle

The core loop resides in the Agent.run() async generator method. Each iteration follows a strict sequence:

1. Prompt Construction: buildInitialPrompt() or buildIterationPrompt() assembles the conversation history, system instructions, and previous tool results.
2. LLM Invocation: callModel() sends the prompt with tools bound, receiving an AIMessage containing potential tool calls.
3. Tool Detection: hasToolCalls() in src/utils/ai-message.ts inspects the response for tool_calls arrays.
4. Execution: If tools are present, AgentToolExecutor.executeAll() runs them sequentially.
5. Result Recording: Raw results append to the scratchpad via scratchpad.addToolResult().
6. Context Management: manageContextThreshold() checks token estimates against CONTEXT_THRESHOLD, clearing oldest results if necessary.

Tool Execution and Result Management

Detecting and Parsing Tool Calls

The loop detects tool calls using the hasToolCalls() utility in src/utils/ai-message.ts. When the LLM returns structured tool invocations, the executor extracts the tool name and arguments from the tool_calls array embedded in the AIMessage.

Sequential Tool Execution with Progress Streaming

The AgentToolExecutor class in src/agent/tool-executor.ts handles execution through two primary methods:

• executeAll(): Iterates over every tool call in the response, invoking executeSingle() for each.
• executeSingle(): Runs the individual tool with progress streaming via createProgressChannel().

// src/agent/tool-executor.ts
async executeSingle(toolCall: ToolCall, metadata: ToolMetadata) {
  const channel = createProgressChannel();
  
  // Tool runs and pushes progress updates
  const result = await tool.run(toolCall.args, { onProgress: channel.push });
  
  return { result, toolCall };
}

The executor emits tool_start, tool_progress, tool_end, and tool_error events, enabling real-time UI updates during long-running financial calculations.

Append-Only Scratchpad Storage

Unlike systems that summarize previous results, Dexter maintains an append-only scratchpad in src/agent/scratchpad.ts. The addToolResult() method records the full JSON-serialized result:

// src/agent/scratchpad.ts
addToolResult(toolCallId: string, toolName: string, result: unknown) {
  this.entries.push({
    type: 'tool_result',
    toolCallId,
    toolName,
    result: JSON.stringify(result),
    timestamp: Date.now()
  });
}

This approach ensures complete auditability and allows the final answer generation to access the full history of tool executions.

Context Management and Iteration Control

Token Threshold Monitoring

After each tool execution, the loop invokes manageContextThreshold() in src/agent/agent.ts. This function estimates the current token count using estimateTokenCount() from src/utils/tokens.ts and compares it against the CONTEXT_THRESHOLD constant.

Anthropic-Style Context Pruning

When the token estimate exceeds the threshold, Dexter implements Anthropic-style context management through Scratchpad.clearOldestToolResults():

// src/agent/scratchpad.ts
clearOldestToolResults(count: number) {
  const toolResults = this.entries.filter(e => e.type === 'tool_result');
  const toClear = toolResults.slice(0, -KEEP_TOOL_USES);
  
  toClear.forEach(entry => {
    entry.cleared = true; // Logical deletion
  });
}

Rather than deleting data, the system marks older entries as cleared. These entries remain in the JSONL log for persistence but are omitted from subsequent iteration prompts. The KEEP_TOOL_USES constant (defined in src/utils/tokens.ts) ensures the most recent tool results remain available for the model's reasoning.

Final Answer Generation

When hasToolCalls() returns false, indicating the model requires no further tools, the loop transitions to generateFinalAnswer() in src/agent/agent.ts. This method:

1. Constructs a rich context using buildFinalAnswerContext() that includes all tool results (including cleared entries)
2. Calls the LLM with useTools = false to prevent further tool binding
3. Emits an answer_start event followed by the final content
4. Yields a done event containing timing metrics and total token usage

Event-Driven Architecture and Usage Example

Dexter's agent loop implements an async generator pattern that yields typed AgentEvent objects, enabling real-time streaming interfaces. The following example demonstrates how to instantiate the agent and consume its event stream:

import { Agent } from './agent/agent.js';
import { InMemoryChatHistory } from './utils/in-memory-chat-history.js';

// Create the agent (auto‑loads all registered tools for the default model)
const agent = Agent.create({ maxIterations: 8 });

// Optional: supply previous user queries for context
const history = new InMemoryChatHistory();
history.addUserMessage('What was Apple’s revenue last quarter?');
history.addUserMessage('How does it compare to Microsoft?');

// Run the agent – consume the async generator
(async () => {
  for await (const event of agent.run('Give me a DCF valuation for Tesla.', history)) {
    switch (event.type) {
      case 'thinking':
        console.log('🤔', event.message);
        break;
      case 'tool_start':
        console.log(`🔧 Starting ${event.tool}...`);
        break;
      case 'tool_progress':
        console.log(`   ↳ ${event.message}`);
        break;
      case 'tool_end':
        console.log(`✅ ${event.tool} finished (${event.duration} ms)`);
        break;
      case 'context_cleared':
        console.log(`🧹 Cleared ${event.clearedCount} old tool results`);
        break;
      case 'answer_start':
        console.log('\n--- Final Answer ---');
        break;
      case 'done':
        console.log('\n', event.answer);
        console.log(`\n🕒 ${event.totalTime} ms | 📦 ${event.tokenUsage?.total ?? 0} tokens`);
        return;
    }
  }
})();

This implementation demonstrates the full lifecycle: initial prompt construction, sequential tool execution with progress streaming, Anthropic-style context pruning, and final answer synthesis.

Summary

• Dexter's agent loop operates as an async generator in src/agent/agent.ts, yielding typed events for real-time UI updates while orchestrating multi-step LLM interactions.
• Tool execution is handled by AgentToolExecutor in src/agent/tool-executor.ts, which runs tools sequentially, streams progress via channels, and emits lifecycle events (tool_start, tool_progress, tool_end, tool_error).
• Result persistence uses an append-only Scratchpad in src/agent/scratchpad.ts that stores full JSON-serialized tool outputs without summarization, ensuring complete auditability.
• Context management implements Anthropic-style pruning through manageContextThreshold() and clearOldestToolResults(), which logically clears old entries from the prompt while retaining them in the JSONL log.
• Iteration control continues until hasToolCalls() returns false or maxIterations is reached, culminating in generateFinalAnswer() which synthesizes the final response using the complete scratchpad history.

Frequently Asked Questions

How does Dexter's agent loop handle multiple tool calls in a single LLM response?

When the LLM returns multiple tool calls in one response, AgentToolExecutor.executeAll() in src/agent/tool-executor.ts iterates over the tool_calls array and invokes executeSingle() for each entry. The executor runs these sequentially rather than in parallel, emitting tool_start and tool_end events for each individual tool to maintain clear execution boundaries and error isolation.

What happens when Dexter's agent loop reaches the maximum iteration limit?

If the loop exhausts the maxIterations count configured in Agent.create() without the LLM returning a tool-call-free response, the system forcibly terminates the iteration cycle and invokes generateFinalAnswer() with the current scratchpad state. This ensures the user receives a synthesized answer even if the model remains in a "tool-calling mode," preventing infinite loops while preserving all gathered data.

How does Dexter prevent duplicate tool executions during the agent loop?

The Scratchpad class implements deduplication through hasExecutedSkill() in src/agent/scratchpad.ts, which tracks skill-based tools to ensure they execute at most once per query. Additionally, canCallTool() enforces per-tool call limits (defaulting to 3 attempts) by checking counters stored in the scratchpad, emitting tool_limit warnings when the threshold is exceeded to prevent redundant API calls.

What is the difference between the iteration prompt and final answer prompt in Dexter?

The iteration prompt—built by buildIterationPrompt()—includes the original query, recent tool results (excluding logically cleared entries), and a formatted usage summary showing attempted calls, but omits older cleared results to stay within token limits. Conversely, the final answer prompt—constructed by buildFinalAnswerContext() in generateFinalAnswer()—incorporates the complete scratchpad history including cleared entries, providing the model with full context for synthesis, and is invoked with useTools = false to prevent further tool binding.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}