Understanding the Scratchpad in Dexter's Agent Loop and Context Management

The scratchpad is Dexter's append-only transaction log that serves as the single source of truth for all tool executions, providing the LLM with complete context while managing token limits through intelligent truncation.

The scratchpad is the central nervous system of the Dexter agent architecture. In the virattt/dexter repository, this component functions as a persistent, query-scoped memory that records every tool call, result, and decision made during an agent's execution loop. Understanding how the scratchpad operates is essential for anyone building or debugging agentic workflows in Dexter.

What Is the Scratchpad in Dexter?

At its core, the scratchpad is an append-only log that maintains a complete record of a single query's execution lifecycle. Unlike volatile memory that resets between iterations, the scratchpad persists tool results and metadata across the entire agent loop, enabling iterative reasoning and context accumulation.

Core Implementation in scratchpad.ts

The scratchpad implementation lives in src/agent/scratchpad.ts. This file defines the Scratchpad class, which manages:

• Tool result storage: Appending JSON lines to .dexter/scratchpad/<timestamp>_<hash>.jsonl
• Usage tracking: Monitoring how many times each tool is invoked
• Similarity detection: Preventing redundant queries by comparing current requests against historical ones
• Context management: Clearing oldest results when token thresholds are exceeded

Three Critical Roles of the Scratchpad in the Agent Loop

The scratchpad serves three distinct but interconnected functions that enable Dexter's autonomous operation.

1. Single Source of Truth for All Work

When a run initiates, createRunContext() in src/agent/run-context.ts instantiates a fresh Scratchpad with the user's query:

import { createRunContext } from './run-context.js';

const ctx = createRunContext('How many shares did AAPL buy in Q3 2023?');
// Scratchpad automatically writes the initial entry:
// { type: 'init', content: query, timestamp: … }

This initialization establishes the scratchpad as the authoritative record for the entire execution. Every subsequent tool execution, intermediate thought, and context modification flows through this central log.

2. Providing Full Context to the LLM

During each iteration of the agent loop, Dexter constructs prompts that include the complete history of tool executions. In src/agent/agent.ts, the agent retrieves formatted context using:

• scratchpad.getToolResults(): Returns all tool outputs as structured strings
• scratchpad.formatToolUsageForPrompt(): Generates a usage summary showing which tools were called and how frequently

const iterationPrompt = buildIterationPrompt(
  query,
  ctx.scratchpad.getToolResults(),           // "### financial_metrics(ticker=AAPL, metric=insider_buy) …"

  ctx.scratchpad.formatToolUsageForPrompt()  // "## Tool Usage This Query …"

);

This "Anthropic-style" approach of feeding the LLM all prior tool outputs enables the agent to perform complex, multi-step reasoning by referencing earlier results without hallucinating intermediate states.

3. Managing Context Size and Tool-Call Limits

Long-running agent loops risk exceeding LLM token limits or falling into infinite tool-call cycles. The scratchpad implements safeguards in src/agent/agent.ts through manageContextThreshold():

// Inside Agent.manageContextThreshold()
if (estimatedContextTokens > CONTEXT_THRESHOLD) {
  const cleared = ctx.scratchpad.clearOldestToolResults(KEEP_TOOL_USES);
  // Emits a ContextClearedEvent so the UI can show what was dropped
}

When the estimated token count exceeds CONTEXT_THRESHOLD, the scratchpad clears the oldest tool results while preserving the most recent KEEP_TOOL_USES entries. This ensures the agent maintains access to relevant recent context without overwhelming the LLM's context window.

Additionally, the scratchpad tracks tool usage frequency and detects query similarity to prevent redundant executions, as implemented in src/agent/scratchpad.ts lines 30-78.

How the Scratchpad Powers the Final Answer

When the agent determines it has sufficient information to respond, generateFinalAnswer() in src/agent/agent.ts leverages the scratchpad's complete record:

const fullContext = buildFinalAnswerContext(ctx.scratchpad);
const finalPrompt = buildFinalAnswerPrompt(query, fullContext);
const { response } = await callLlm(finalPrompt, { model, systemPrompt });

The buildFinalAnswerContext() function (located in src/agent/final-answer-context.ts) transforms the scratchpad's stored tool results into a formatted context string. This ensures the final synthesis has access to every tool output collected during the execution, preventing information loss between the iterative phase and the final response generation.

Code Examples: Working with the Scratchpad

Adding a Tool Result

When AgentToolExecutor completes a tool call, it appends the result to the scratchpad:

ctx.scratchpad.addToolResult(
  'financial_metrics',
  { ticker: 'AAPL', metric: 'insider_buy' },
  '{"shares": 120000}'   // raw JSON string from the tool
);

The result is stored as a JSON line in .dexter/scratchpad/<timestamp>_<hash>.jsonl and remains available via getToolResults().

Automatic Context Clearing

The agent automatically manages token budgets by clearing old results:

// Inside Agent.manageContextThreshold()
if (estimatedContextTokens > CONTEXT_THRESHOLD) {
  const cleared = ctx.scratchpad.clearOldestToolResults(KEEP_TOOL_USES);
  // Emits a ContextClearedEvent so the UI can show what was dropped
}

Key Files and Their Roles

File	Role
src/agent/scratchpad.ts	Implements the append-only log, tool-result storage, usage tracking, similarity detection, and context-clearing logic.
src/agent/run-context.ts	Creates a RunContext that bundles the query, a fresh Scratchpad, and a token counter for each run.
src/agent/agent.ts	Orchestrates the agent loop, uses the scratchpad to feed tool results to the LLM, clears old results, and builds the final answer.
src/agent/final-answer-context.ts	Converts the scratchpad's stored tool results into the formatted context string used for the final answer generation.

Summary

• The scratchpad is an append-only transaction log that serves as the single source of truth for all tool executions during a Dexter agent run.
• It provides the LLM with complete context by feeding all prior tool outputs into each iteration prompt, enabling complex multi-step reasoning.
• The scratchpad manages token limits through automatic clearing of oldest tool results when context thresholds are exceeded, while preserving recent critical data.
• Located in src/agent/scratchpad.ts, it integrates with run-context.ts for initialization and agent.ts for orchestration, forming the backbone of Dexter's context management system.

Frequently Asked Questions

How does the scratchpad prevent token limit errors?

The scratchpad implements a threshold-based clearing mechanism in src/agent/agent.ts. When manageContextThreshold() detects that estimated tokens exceed CONTEXT_THRESHOLD, it invokes clearOldestToolResults(KEEP_TOOL_USES) to remove the oldest entries while preserving the most recent tool outputs. This ensures the LLM receives relevant recent context without exceeding model token limits.

Is the scratchpad persisted between different queries?

No, the scratchpad is query-scoped and ephemeral. Each new query triggers createRunContext() in src/agent/run-context.ts, which instantiates a fresh Scratchpad instance. While the scratchpad writes to disk in .dexter/scratchpad/<timestamp>_<hash>.jsonl for the duration of the run, it does not persist across separate query executions.

How does the scratchpad track tool usage limits?

The scratchpad maintains internal counters for each tool invocation, as implemented in src/agent/scratchpad.ts (lines 30-78). It tracks how many times each tool has been called during the current run, warns when approaching predefined limits, and detects query similarity to prevent redundant or cyclical tool calls that could waste resources or cause infinite loops.

What happens to scratchpad data when generating the final answer?

When the agent transitions to final answer generation, generateFinalAnswer() in src/agent/agent.ts calls buildFinalAnswerContext(ctx.scratchpad) to retrieve the complete record of all tool outputs. The scratchpad's getToolCallRecords() method provides the structured history needed to synthesize a comprehensive response that incorporates every piece of information gathered during the iterative tool-calling phase.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}