# Context Budget and Compaction in Apache Maka: Managing LLM Token Limits for Long-Running Agents

> Learn about context budget and compaction in Apache Maka. These features manage LLM token limits for long-running agents, preserving auditable event history. Read more!

- Repository: [The Apache Software Foundation/maka](https://github.com/apache/maka)
- Tags: deep-dive
- Published: 2026-09-09

---

**Context budget and compaction are Apache Maka's dual mechanisms for keeping long-lived agent sessions within LLM token limits while preserving an immutable, auditable event history.**

Apache Maka is an open-source framework for building persistent AI agents. As these agents accumulate interaction history, their **RuntimeEvent log** grows indefinitely. Since LLMs have fixed context windows, Maka cannot simply send the entire history on every request. Instead, the runtime uses **context budget policies** to decide what fits, and **compaction** to create lossy summaries when the budget is exceeded.

## What Is Context Budget in Apache Maka?

The **context budget** is a policy-driven limit on how many tokens may be sent to the LLM provider per request. It lives in [`packages/runtime/src/context-budget.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/context-budget.ts) and [`packages/runtime/src/context-budget-policy.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/context-budget-policy.ts).

A budget policy defines:

- **`capacity`**: Maximum tokens the model receives in one request
- **`charsPerToken`**: Estimation ratio for sizing content
- **`activeToolResultPrune`**: Rules for aggressively trimming tool outputs when near the limit

When the runtime builds a request, `applyRuntimeEventContextBudget` checks whether the accumulated history plus expected output would exceed capacity. The function returns a **`ContextBudgetDiagnostic`** that records decisions such as:

| Decision | Meaning |
|----------|---------|
| `replaced` | Content was substituted with a compaction checkpoint |
| `unchanged` | All content fit within budget |
| `failedOpen` | Compaction was attempted but rejected by the provider |

These diagnostics are attached to every request and drive telemetry and cost accounting.

## What Is Compaction in Apache Maka?

**Compaction** is a *lossy projection* of the RuntimeEvent log that creates a **checkpoint** summarizing a safe prefix of history. The implementation spans [`packages/runtime/src/history-compaction.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/history-compaction.ts) and [`packages/runtime/src/ai-sdk-compaction.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/ai-sdk-compaction.ts).

Unlike truncation, compaction preserves the immutable log. It produces a **continuation view** that subsequent requests use instead of the raw prefix. The design document *LLM Compaction Events Log Projection* explains this mental model: the log stays append-only; compaction builds a projected context for the provider.

The compactor supports two paths:

- **Text summary** (fallback): A human-readable condensation of events
- **Provider-native compaction**: For example, OpenAI's `openai.compaction` custom part

## How Context Budget and Compaction Work Together

The integration follows a six-step flow as implemented in the Apache Maka source code:

1. **Request assembly** — `AiSdkTurn` builds the provider request from current session state

2. **Budget consultation** — `applyRuntimeEventContextBudget` compares token usage against policy capacity

3. **Compaction trigger** — If the budget requires it, `AiSdkCompaction` executes a compact operation

4. **Checkpoint persistence** — A V2-text or V3-native checkpoint is written via [`history-compact-checkpoint.ts`](https://github.com/apache/maka/blob/main/history-compact-checkpoint.ts)

5. **Projected context reads** — Future turns combine the checkpoint with the raw tail of events

6. **Diagnostic recording** — Every decision becomes a `ContextBudgetDiagnostic` in [`context-diagnostics.ts`](https://github.com/apache/maka/blob/main/context-diagnostics.ts)

## Implementing Context Budget and Compaction: Code Examples

### Define a Context Budget Policy

```typescript
import { ContextBudgetPolicy } from "./context-budget-policy";

const budgetPolicy: ContextBudgetPolicy = {
  charsPerToken: 4,
  capacity: 4000,
  activeToolResultPrune: { maxSize: 2000, keepRecent: 2 },
};

```

### Apply the Budget to Runtime Context

```typescript
import { applyRuntimeEventContextBudget } from "./context-budget";

const { diagnostic, projectedContext } = applyRuntimeEventContextBudget(
  priorRuntimeContext,
  budgetPolicy
);

```

### Execute Compaction When Required

```typescript
import { AiSdkCompaction } from "./ai-sdk-compaction";

if (diagnostic?.compactionDecisions?.some(d => d.decision === "replaced")) {
  const compactor = new AiSdkCompaction({
    contextBudget: budgetPolicy,
    // provider connection, model configuration, etc.
  });
  const { checkpoint, notes } = await compactor.run();
  // checkpoint persisted; notes used for telemetry
}

```

### Handle Fail-Open Diagnostics

```typescript
if (turnResult.contextBudget?.compactionDecisions?.[0].decision === "failedOpen") {
  console.warn(
    "Compaction failed:",
    turnResult.contextBudget?.compactionDecisions?.[0].failOpenReason
  );
}

```

## Key Source Files for Context Budget and Compaction

| File | Purpose |
|------|---------|
| [`packages/runtime/src/context-budget.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/context-budget.ts) | Core budgeting algorithm and diagnostic helpers |
| [`packages/runtime/src/context-budget-policy.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/context-budget-policy.ts) | Policy data structures and validation |
| [`packages/runtime/src/ai-sdk-compaction.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/ai-sdk-compaction.ts) | Compaction orchestration with provider integration |
| [`packages/runtime/src/history-compaction.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/history-compaction.ts) | Checkpoint construction and fail-open handling |
| [`docs/architecture/llm-compaction-events-log-projection-draft.md`](https://github.com/apache/maka/blob/main/docs/architecture/llm-compaction-events-log-projection-draft.md) | Design rationale: compaction as projection |

## Summary

- **Context budget** enforces token limits per request through configurable policies in [`context-budget.ts`](https://github.com/apache/maka/blob/main/context-budget.ts)

- **Compaction** creates lossy checkpoints that preserve immutable history while fitting within budget

- **Projected context** combines checkpoints with raw event tails for unlimited session length

- **Diagnostics** in `ContextBudgetDiagnostic` enable observability and graceful degradation via fail-open handling

- The architecture guarantees **replayability** and **auditability** by keeping the RuntimeEvent log append-only

## Frequently Asked Questions

### What happens when compaction fails in Apache Maka?

The runtime enters **fail-open** mode. The `ContextBudgetDiagnostic` records `failedOpen` with a `failOpenReason`, and the request continues with whatever context fits. This prevents session crashes while alerting operators via telemetry.

### Can I customize how Apache Maka compacts session history?

Yes. The `ContextBudgetPolicy` in [`context-budget-policy.ts`](https://github.com/apache/maka/blob/main/context-budget-policy.ts) accepts custom pruning rules. For provider-native compaction, extend `AiSdkCompaction` in [`ai-sdk-compaction.ts`](https://github.com/apache/maka/blob/main/ai-sdk-compaction.ts) to integrate additional LLM services beyond OpenAI.

### Does compaction lose information permanently?

No. Compaction is a **projection**, not deletion. The original RuntimeEvent log remains immutable in storage. Only the LLM's *view* of history is reduced. Full replay and audit remain possible by reading the complete log.

### How do I monitor context budget decisions in production?

Every turn includes `ContextBudgetDiagnostic` data. Aggregate `compactionDecisions` by decision type (`replaced`, `unchanged`, `failedOpen`) to track compression ratios, fail-open rates, and estimated cost savings from compaction versus raw context transmission.