Context Budget and Compaction in Apache Maka: Managing LLM Token Limits for Long-Running Agents

Context budget and compaction are Apache Maka's dual mechanisms for keeping long-lived agent sessions within LLM token limits while preserving an immutable, auditable event history.

Apache Maka is an open-source framework for building persistent AI agents. As these agents accumulate interaction history, their RuntimeEvent log grows indefinitely. Since LLMs have fixed context windows, Maka cannot simply send the entire history on every request. Instead, the runtime uses context budget policies to decide what fits, and compaction to create lossy summaries when the budget is exceeded.

What Is Context Budget in Apache Maka?

The context budget is a policy-driven limit on how many tokens may be sent to the LLM provider per request. It lives in packages/runtime/src/context-budget.ts and packages/runtime/src/context-budget-policy.ts.

A budget policy defines:

• capacity: Maximum tokens the model receives in one request
• charsPerToken: Estimation ratio for sizing content
• activeToolResultPrune: Rules for aggressively trimming tool outputs when near the limit

When the runtime builds a request, applyRuntimeEventContextBudget checks whether the accumulated history plus expected output would exceed capacity. The function returns a ContextBudgetDiagnostic that records decisions such as:

Decision	Meaning
replaced	Content was substituted with a compaction checkpoint
unchanged	All content fit within budget
failedOpen	Compaction was attempted but rejected by the provider

These diagnostics are attached to every request and drive telemetry and cost accounting.

What Is Compaction in Apache Maka?

Compaction is a lossy projection of the RuntimeEvent log that creates a checkpoint summarizing a safe prefix of history. The implementation spans packages/runtime/src/history-compaction.ts and packages/runtime/src/ai-sdk-compaction.ts.

Unlike truncation, compaction preserves the immutable log. It produces a continuation view that subsequent requests use instead of the raw prefix. The design document LLM Compaction Events Log Projection explains this mental model: the log stays append-only; compaction builds a projected context for the provider.

The compactor supports two paths:

• Text summary (fallback): A human-readable condensation of events
• Provider-native compaction: For example, OpenAI's openai.compaction custom part

How Context Budget and Compaction Work Together

The integration follows a six-step flow as implemented in the Apache Maka source code:

1. Request assembly — AiSdkTurn builds the provider request from current session state

2. Budget consultation — applyRuntimeEventContextBudget compares token usage against policy capacity

3. Compaction trigger — If the budget requires it, AiSdkCompaction executes a compact operation

4. Checkpoint persistence — A V2-text or V3-native checkpoint is written via history-compact-checkpoint.ts

5. Projected context reads — Future turns combine the checkpoint with the raw tail of events

6. Diagnostic recording — Every decision becomes a ContextBudgetDiagnostic in context-diagnostics.ts

Implementing Context Budget and Compaction: Code Examples

Define a Context Budget Policy

import { ContextBudgetPolicy } from "./context-budget-policy";

const budgetPolicy: ContextBudgetPolicy = {
  charsPerToken: 4,
  capacity: 4000,
  activeToolResultPrune: { maxSize: 2000, keepRecent: 2 },
};

Apply the Budget to Runtime Context

import { applyRuntimeEventContextBudget } from "./context-budget";

const { diagnostic, projectedContext } = applyRuntimeEventContextBudget(
  priorRuntimeContext,
  budgetPolicy
);

Execute Compaction When Required

import { AiSdkCompaction } from "./ai-sdk-compaction";

if (diagnostic?.compactionDecisions?.some(d => d.decision === "replaced")) {
  const compactor = new AiSdkCompaction({
    contextBudget: budgetPolicy,
    // provider connection, model configuration, etc.
  });
  const { checkpoint, notes } = await compactor.run();
  // checkpoint persisted; notes used for telemetry
}

Handle Fail-Open Diagnostics

if (turnResult.contextBudget?.compactionDecisions?.[0].decision === "failedOpen") {
  console.warn(
    "Compaction failed:",
    turnResult.contextBudget?.compactionDecisions?.[0].failOpenReason
  );
}

Key Source Files for Context Budget and Compaction

File	Purpose
packages/runtime/src/context-budget.ts	Core budgeting algorithm and diagnostic helpers
packages/runtime/src/context-budget-policy.ts	Policy data structures and validation
packages/runtime/src/ai-sdk-compaction.ts	Compaction orchestration with provider integration
packages/runtime/src/history-compaction.ts	Checkpoint construction and fail-open handling
docs/architecture/llm-compaction-events-log-projection-draft.md	Design rationale: compaction as projection

Summary

• Context budget enforces token limits per request through configurable policies in context-budget.ts

• Compaction creates lossy checkpoints that preserve immutable history while fitting within budget

• Projected context combines checkpoints with raw event tails for unlimited session length

• Diagnostics in ContextBudgetDiagnostic enable observability and graceful degradation via fail-open handling

• The architecture guarantees replayability and auditability by keeping the RuntimeEvent log append-only

Frequently Asked Questions

What happens when compaction fails in Apache Maka?

The runtime enters fail-open mode. The ContextBudgetDiagnostic records failedOpen with a failOpenReason, and the request continues with whatever context fits. This prevents session crashes while alerting operators via telemetry.

Can I customize how Apache Maka compacts session history?

Yes. The ContextBudgetPolicy in context-budget-policy.ts accepts custom pruning rules. For provider-native compaction, extend AiSdkCompaction in ai-sdk-compaction.ts to integrate additional LLM services beyond OpenAI.

Does compaction lose information permanently?

No. Compaction is a projection, not deletion. The original RuntimeEvent log remains immutable in storage. Only the LLM's view of history is reduced. Full replay and audit remain possible by reading the complete log.

How do I monitor context budget decisions in production?

Every turn includes ContextBudgetDiagnostic data. Aggregate compactionDecisions by decision type (replaced, unchanged, failedOpen) to track compression ratios, fail-open rates, and estimated cost savings from compaction versus raw context transmission.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}