How OpenCode Implements Session Compaction and Memory Optimization

OpenCode detects context window overflow in processor.ts, triggers compaction via the /session/:sessionID/compact endpoint, and executes a summarization LLM call in compaction.ts to replace detailed history with concise summaries while pruning old tool output.

OpenCode is an open-source AI coding assistant that manages long-running conversations through an intelligent session compaction system. As conversations grow and approach model context limits, the system automatically optimizes memory usage by summarizing historical interactions. This article examines the exact implementation details found in the anomalyco/opencode repository, covering overflow detection, compaction triggering, and memory pruning mechanisms.

Detecting Context Window Overflow

OpenCode monitors token consumption after every LLM step to determine when the conversation approaches the model's context window. The detection logic resides in packages/opencode/src/session/compaction.ts within the isOverflow function.

Token Counting and Safety Buffers

The system calculates total token usage by aggregating input, output, and cache tokens:

const count = input.tokens.total ||
              input.tokens.input + input.tokens.output +
              input.tokens.cache.read + input.tokens.cache.write;

To prevent hitting the absolute limit, OpenCode reserves a safety buffer (defaulting to 20,000 tokens or the model's maximum output token count). The usable token quota is calculated as either the model's context limit minus the buffer, or the input limit minus the buffer, depending on the model's configuration.

Processor Integration

After each processing step, packages/opencode/src/session/processor.ts evaluates whether compaction is necessary:

if (await SessionCompaction.isOverflow({ tokens: usage.tokens, model: input.model })) {
  needsCompaction = true;          // will break the streaming loop
}

This check occurs at lines 82-84 of the processor, setting a flag that ultimately returns "compact" to the orchestrator.

Triggering the Compaction Process

Once overflow is detected, OpenCode transitions from detection to action through either automatic or manual triggers.

HTTP Endpoint for Manual Compaction

Developers can manually initiate compaction by calling the REST endpoint defined in packages/opencode/src/server/routes/session.ts:

// POST /session/:sessionID/compact
await SessionCompaction.create({
  sessionID,
  agent: currentAgent,
  model: { providerID: body.providerID, modelID: body.modelID },
  auto: body.auto,
});

This route accepts parameters specifying the model provider and whether the compaction should run automatically.

Automatic Triggering via Session Loop

When the processor detects overflow, it returns "compact" to SessionPrompt.loop (the top-level orchestrator). The loop then automatically invokes:

await SessionCompaction.create({
  sessionID: input.sessionID,
  agent: input.agent,
  model: input.model,
  auto: true,
});

This seamless integration ensures that long-running sessions remain within context limits without manual intervention.

Executing Session Compaction

The actual compaction logic resides in packages/opencode/src/session/compaction.ts, which handles message creation, prompt construction, and LLM execution.

Creating Synthetic Messages

SessionCompaction.create generates a synthetic user message containing a part of type "compaction":

await Session.updateMessage({
  id: Identifier.ascending("message"),
  role: "user",
  model: input.model,
  sessionID: input.sessionID,
  agent: input.agent,
  time: { created: Date.now() },
});

await Session.updatePart({
  id: Identifier.ascending("part"),
  messageID: msg.id,
  sessionID: msg.sessionID,
  type: "compaction",
  auto: input.auto,
});

This message signals the system to enter compaction mode during the next processing loop.

Building the Compaction Prompt

The compaction processor constructs a prompt using either a default template or plugin-augmented content:

const defaultPrompt = `Provide a detailed prompt for continuing our conversation above…
---`;

const compacting = await Plugin.trigger(
  "experimental.session.compacting",
  { sessionID: input.sessionID },
  { context: [], prompt: undefined },
);

const promptText = compacting.prompt ?? [defaultPrompt, ...compacting.context].join("\n\n");

Plugins can inject domain-specific context or completely replace the summarization prompt via the experimental.session.compacting hook.

LLM Execution and Summary Storage

The processor streams the compaction request to the LLM:

const stream = await LLM.stream({
  model: input.model,
  prompt: promptText,
  // ... other options
});

The resulting assistant message is marked as a summary (summary: true). Upon completion, the system publishes a session.compacted event:

Bus.publish(Event.Compacted, { sessionID: input.sessionID });
return "continue";

Memory Optimization Mechanisms

Beyond summarization, OpenCode employs aggressive pruning strategies to reclaim token budget from outdated tool outputs.

Pruning Old Tool Output

The SessionCompaction.prune function (called before new compactions) walks backward through the session history, aggregating tool-call output tokens. When the accumulated total exceeds PRUNE_PROTECT (a configurable threshold), it marks the oldest tool outputs as compacted:

if (total > PRUNE_PROTECT) {
  pruned += estimate;
  toPrune.push(part);
}
// ...
part.state.time.compacted = Date.now();
await Session.updatePart(part);

This timestamp (state.time.compacted) signals the prompt builder to exclude these parts from future LLM requests, effectively removing their token cost from the context window.

Configuration Options

Compaction behavior is controlled via the compaction configuration block in packages/opencode/src/config/config.ts:

compaction: {
  auto: z.boolean().optional().describe("Enable automatic compaction (default true)"),
  prune: z.boolean().optional().describe("Enable pruning of old tool output (default true)"),
  reserved: z.number().int().min(0).optional()
                 .describe("Token buffer reserved for the compaction step"),
}

Additionally, setting the environment variable OPENCODE_DISABLE_AUTOCOMPACT forcibly disables automatic compaction across all sessions.

Practical Implementation Examples

Automatic Compaction (Default)

The default configuration requires no explicit code. The SessionPrompt.loop automatically handles overflow:

// The processor automatically detects overflow.
// No explicit call from the consumer is required.
await SessionPrompt.loop({ sessionID: "abc123" });
// When the token limit is hit, the loop will internally call
// SessionCompaction.create → compaction → summary.

Manual Compaction via API

Force immediate compaction via the REST API:

curl -X POST https://api.opencode.ai/project/42/session/abc123/compact \
     -H "Authorization: Bearer <token>" \
     -d '{"providerID":"openai","modelID":"gpt-4","auto":true}'

This inserts a compaction message into the session history, triggering the summarization process on the next loop iteration.

Custom Compaction Prompts via Plugin

Extend the compaction logic with domain-specific context:

// plugins/compaction.ts
export default {
  "experimental.session.compacting": async (input, output) => {
    // Add domain‑specific context before the default template
    output.context.push("Project: my‑web‑app\nCurrent branch: feature/login");
    // Optionally replace the whole prompt
    // output.prompt = "Summarize only the recent API changes.";
  },
};

When the compaction runs, the plugin’s output.context is concatenated with the default template (see Plugin.trigger call in compaction.ts).

Summary

• Overflow Detection: The isOverflow function in packages/opencode/src/session/compaction.ts monitors token counts against configurable safety buffers after every LLM step in processor.ts.
• Automatic Triggering: When tokens exceed the usable quota (context limit minus reserved buffer), SessionPrompt.loop automatically invokes SessionCompaction.create to initiate summarization.
• Synthetic Message Creation: Compaction inserts a user message with a "compaction" part type, signaling the system to enter summarization mode.
• Configurable Pruning: The prune function removes outdated tool output by marking parts with state.time.compacted, reclaiming token budget before summarization occurs.
• Plugin Extensibility: The experimental.session.compacting hook allows custom prompts and context injection via the plugin system.

Frequently Asked Questions

What triggers session compaction in OpenCode?

Session compaction triggers when the accumulated token count in a conversation approaches the model's context window limit. Specifically, the isOverflow function in packages/opencode/src/session/compaction.ts compares the current token total (including input, output, and cache tokens) against a usable quota calculated as the model's context limit minus a configurable reserved buffer. When needsCompaction is set to true in processor.ts (lines 82-84), the system initiates the compaction workflow.

How does OpenCode determine which messages to prune during memory optimization?

OpenCode prunes messages by walking backward through the session history and aggregating tool-call output tokens. The SessionCompaction.prune function in compaction.ts accumulates token estimates until the total exceeds PRUNE_PROTECT, a configurable threshold. Once exceeded, it marks the oldest tool output parts with state.time.compacted = Date.now(), signaling the prompt builder to exclude these parts from future LLM requests. This process specifically targets tool outputs rather than user or assistant messages to preserve conversation continuity.

Can I disable automatic session compaction in OpenCode?

Yes, automatic compaction can be disabled through configuration or environment variables. Set compaction.auto to false in the configuration schema defined in packages/opencode/src/config/config.ts, or set the environment variable OPENCODE_DISABLE_AUTOCOMPACT to forcibly disable the feature across all sessions. When disabled, the isOverflow function returns false immediately, preventing the automatic triggering mechanism in SessionPrompt.loop from initiating compaction.

How can I customize the summarization prompt used during compaction?

Customize the compaction prompt by implementing the experimental.session.compacting plugin hook. Create a plugin that exports this hook and modify the output.context array to inject domain-specific context before the default template, or set output.prompt to completely replace the summarization instructions. The SessionCompaction.process function in compaction.ts calls Plugin.trigger for this hook, concatenating any plugin-provided context with the default prompt template before streaming to the LLM.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}