How the Codebuff Context-Pruner Mechanism Manages Token Limits

The Codebuff context-pruner mechanism automatically compresses conversation history using a two-phase approach—first checking if pruning is necessary, then generating a compact summary—to keep token counts under a configurable 200,000-token budget while preserving critical context.

The context-pruner is a hidden sub-agent in the CodebuffAI/codebuff repository that executes automatically before every step of a parent agent. Its primary responsibility is managing the token budget of ongoing conversations, ensuring that cumulative token counts—including file-tree data—remain within configurable limits while retaining the most important contextual information.

Overview of the Context-Pruner Architecture

The mechanism operates through two distinct phases:

1. Quick-exit check – Evaluates whether the current conversation state already satisfies token constraints, allowing immediate return if no pruning is needed.
2. Summarisation mode – Constructs a compact "conversation summary" message that replaces the full message history, significantly reducing token count while preserving essential context.

Both phases rely on specialized helper utilities defined in agents/context-pruner.ts, including countTokensJson for accurate token counting, truncateFileTreeBasedOnTokenBudget for managing file-tree size, and a fudge factor that triggers pruning slightly before hitting the hard limit.

Phase 1: Token-Budget Evaluation and Quick-Exit Logic

Before entering expensive summarisation operations, the context-pruner performs a lightweight check to determine if the current state is already within acceptable bounds.

The evaluation relies on three key parameters defined in agents/context-pruner.ts:

const maxContextLength: number = params?.maxContextLength ?? 200_000;
const TOKEN_COUNT_FUDGE_FACTOR = 1000; // Line 23

The quick-exit condition (around lines 22-28) checks:

if (agentState.contextTokenCount + TOKEN_COUNT_FUDGE_FACTOR <= maxContextLength && !cacheWillMiss) {
  // No pruning needed – strip internal tags and return messages
  yield { toolName: 'set_messages', input: { messages: currentMessages }, includeToolCall: false };
  return;
}

Key components:

• maxContextLength – The hard token limit, defaulting to 200,000 tokens.
• TOKEN_COUNT_FUDGE_FACTOR – A 1,000-token safety margin that triggers pruning early to prevent edge-case overruns.
• cacheWillMiss – A heuristic (lines 84-106) that forces pruning when the user prompt is older than the LLM provider's cache window (5 minutes by default), ensuring cache efficiency even if token counts are technically within limits.

If neither condition requires action, the agent proceeds to summarisation mode.

Phase 2: Summarisation Mode Implementation

When the quick-exit check fails, the context-pruner enters summarisation mode to compress the conversation history into a compact representation.

Preparing the Conversation History

The preparation phase involves several cleanup operations (lines 68-186):

1. Strip recent tags – Removes the most recent INSTRUCTIONS_PROMPT and SUBAGENT_SPAWN tags to prevent circular references.
2. Retrieve existing summaries – Extracts any previous <conversation_summary> blocks to avoid duplicating work (lines 136-164).
3. Filter internal messages – Removes messages containing internal tags (STEP_PROMPT, INSTRUCTIONS_PROMPT, etc.) and prior summaries (lines 170-186).

Building the Compact Summary

For each remaining message, the agent applies role-specific compression strategies:

User messages (lines 216-229):

• Truncates long text using truncateLongText (80% prefix, 20% suffix) implemented at lines 29-43
• Adds [with image(s)] flag if image parts are present

Assistant messages:

• Concatenates plain-text parts
• Removes tool call artifacts and internal formatting

The resulting summary message replaces the entire conversation history, dramatically reducing token count while preserving the semantic content required for subsequent agent steps.

Helper Utilities and Token Management

The context-pruner relies on several specialized utilities in agents/context-pruner.ts:

• countTokensJson – Accurately counts tokens in JSON-serialized message objects using the appropriate tokenizer for the target LLM.
• truncateFileTreeBasedOnTokenBudget – Intelligently truncates file-tree representations to fit within remaining token budgets, prioritizing recently accessed or explicitly mentioned files.
• truncateLongText – Implements the 80/20 truncation strategy (lines 29-43), preserving the beginning and end of long text blocks while eliding the middle.

The fudge factor (1,000 tokens) serves as a critical safety mechanism, ensuring that even with estimation errors or sudden token spikes, the conversation never exceeds the provider's hard context window.

Summary

• The context-pruner mechanism in Codebuff operates as a hidden sub-agent that runs before every parent agent step to enforce token limits.
• It uses a two-phase approach: a quick-exit check (comparing current tokens against a 200,000 default limit with a 1,000-token fudge factor) and a summarisation mode that compresses history.
• The summarisation mode strips internal tags, retrieves existing summaries, filters system messages, and applies role-specific compression (80/20 truncation for user messages).
• Helper utilities like countTokensJson, truncateFileTreeBasedOnTokenBudget, and truncateLongText provide accurate token counting and intelligent content reduction.
• A cache-miss heuristic forces pruning when prompts age beyond the LLM provider's cache window (5 minutes), optimizing both token usage and cache efficiency.

Frequently Asked Questions

What triggers the context-pruner to start summarizing?

The context-pruner enters summarisation mode when either the conversation token count plus a 1,000-token fudge factor exceeds the maxContextLength (default 200,000), or when the cacheWillMiss heuristic detects that the user prompt is older than the LLM provider's 5-minute cache window. If neither condition is met, the quick-exit check returns the messages unchanged.

How does the context-pruner decide what information to keep?

The mechanism prioritizes semantic relevance through a multi-step filtering process. It first strips recent internal tags like INSTRUCTIONS_PROMPT and SUBAGENT_SPAWN, then retrieves existing <conversation_summary> blocks to avoid duplication. For user messages, it applies 80/20 truncation (preserving 80% of the prefix and 20% of the suffix) using the truncateLongText utility, while assistant messages have tool artifacts removed but plain text preserved.

What is the default token limit for Codebuff conversations?

According to the source code in agents/context-pruner.ts, the default maxContextLength is 200,000 tokens. This value is configurable via the params object passed to the pruner, but the 200,000 default provides a safe buffer for most LLM provider context windows. The system also employs a 1,000-token fudge factor that triggers pruning at 199,000 tokens to prevent edge-case overruns.

How does the fudge factor prevent token limit errors?

The TOKEN_COUNT_FUDGE_FACTOR (set to 1,000 tokens at line 23 of agents/context-pruner.ts) acts as a safety margin that triggers the summarisation process before the conversation actually hits the hard maxContextLength limit. This compensates for token estimation inaccuracies, sudden spikes in file-tree data, or message serialization overhead. By initiating compression at 199,000 tokens instead of 200,000, the system ensures the final payload never exceeds provider limits even under variable conditions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}