# How Mako Manages Context and Token Budgets for LLM Interactions: A Deep Dive into the Runtime Budget System

> Discover how Mako manages LLM context and token budgets with its innovative Runtime Budget System. Learn about token estimation, pruning, and history compaction for efficient AI interactions.

- Repository: [The Apache Software Foundation/maka](https://github.com/apache/maka)
- Tags: deep-dive
- Published: 2026-09-06

---

**Mako applies a functional context-budget policy to its immutable Runtime Event Log through three stages: token estimation, stale-tool-result pruning, and history compaction, producing a projection that preserves semantics while keeping requests within provider limits.**

The Apache Mako runtime (from the `apache/maka` repository) treats **context and token budget management** as a pure-functional transformation over an immutable event history. Rather than mutating the persisted ledger, the runtime creates temporary views of the interaction history that fit within configured limits. This design ensures **replay correctness** while preventing LLM requests from exceeding provider-imposed token ceilings.

## The Three-Stage Budget Pipeline

Mako's budget system operates in distinct phases, each implemented in specific source files with clear responsibilities.

### Stage 1: Token Estimation

Before any pruning occurs, Mako **estimates token consumption** using a configurable character-to-token ratio. The default assumes **4 characters per token**, matching common LLM tokenization behavior.

The estimation logic lives in [`packages/runtime/src/context-budget-helpers.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/context-budget-helpers.ts):

```typescript
// estimateTokens @ L35-L38
export function estimateTokens(text: string, charsPerToken: number): number {
  return Math.ceil(utf8ByteLength(text) / charsPerToken);
}

```

This lightweight estimation enables the runtime to make budget decisions without invoking expensive tokenizers for every projection.

### Stage 2: Stale Tool Result Pruning

Tool results from previous turns often consume substantial tokens without contributing to the current conversation semantics. Mako's **`staleToolResultPrune`** policy removes these oversized artifacts.

The policy configuration appears in [`packages/runtime/src/context-budget.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/context-budget.ts) at lines 74-76:

```typescript
interface ContextBudgetPolicy {
  staleToolResultPrune?: {
    enabled: boolean;
    maxChars: number;  // tool results exceeding this are pruned
  };
}

```

**Critical architectural note**: This pruning is **replay-only**—it never mutates the persisted Runtime Event Log. The original events remain intact for recovery and audit purposes.

### Stage 3: History Compaction

When simple pruning insufficient, Mako applies **history compaction** via `applyRuntimeEventHistoryCompactNarrow`. This configurable policy replaces old events with semantically equivalent placeholders, bounding log size while preserving turn continuity.

## The Core Budget Function: applyRuntimeEventContextBudget

The orchestrating function `applyRuntimeEventContextBudget` (lines 13-18 in [`context-budget.ts`](https://github.com/apache/maka/blob/main/context-budget.ts)) implements the complete pipeline:

```typescript
export function applyRuntimeEventContextBudget(
  events: readonly RuntimeEvent[],
  policy: ContextBudgetPolicy | undefined,
): BudgetedRuntimeContext | undefined {
  // ①  Determine which optimizations are active
  const pruneEnabled = policy?.staleToolResultPrune?.enabled === true;
  const historyCompactEnabled = policy?.historyCompact?.enabled === true;
  if (!pruneEnabled && !historyCompactEnabled) return undefined;

  // ②  Pre-compute token estimate for diagnostics
  const charsPerToken = policy?.charsPerToken ?? 4;
  const estimatedTokensBefore = estimateRuntimeEventsTokens(events, charsPerToken);

  // ③  Apply history compaction (stale pruning handled upstream)
  const compacted = applyRuntimeEventHistoryCompactNarrow(
    events,
    policy?.historyCompact,
    charsPerToken,
  );

  // ④  Build comprehensive diagnostic record
  const diagnostic: ContextBudgetDiagnostic = {
    enabled: true,
    policyName: policy?.name,
    estimatedTokensBefore,
    estimatedTokensAfter: estimateRuntimeEventsTokens(compacted.events, charsPerToken),
    keptTurns: new Set(compacted.events.map(runtimeEventTurnKey)).size,
    droppedTurns: /* calculated from difference */,
    keptEvents: compacted.events.length,
    droppedEvents: events.length - compacted.events.length,
    ...compacted.diagnosticPatch,
  };

  return { events: compacted.events, diagnostic, /* ... */ };
}

```

The function returns **budgeted events** (the pruned, compacted projection) plus **diagnostics** for observability.

## From Budgeted Events to Provider Messages

The `events` array from `applyRuntimeEventContextBudget` feeds directly into provider-specific message construction. In [`packages/runtime/src/ai-sdk-backend.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/ai-sdk-backend.ts), the backend:

1. Receives the budgeted `RuntimeEvent` array
2. Projects events into provider-native message formats
3. Submits the resulting request to the LLM provider

This separation of concerns—budget logic in [`context-budget.ts`](https://github.com/apache/maka/blob/main/context-budget.ts), provider adaptation in [`ai-sdk-backend.ts`](https://github.com/apache/maka/blob/main/ai-sdk-backend.ts)—enables Mako to support multiple LLM backends with consistent budget semantics.

## Configuring Context Budget Policies

Clients supply `ContextBudgetPolicy` objects when initializing turns. A typical configuration balancing quality and cost:

```typescript
import { ContextBudgetPolicy } from '@maka/runtime';

const budgetPolicy: ContextBudgetPolicy = {
  name: "production-budget",
  charsPerToken: 4,
  staleToolResultPrune: { 
    enabled: true, 
    maxChars: 20000      // Remove tool results >20KB characters
  },
  activeToolResultPrune: { 
    enabled: true, 
    maxTokens: 2048      // Cap active results at ~2K tokens
  },
  historyCompact: { 
    enabled: true, 
    maxEvents: 500        // Compact when history exceeds 500 events
  },
};

```

The runtime kernel—entry point at [`runtime-kernel.ts`](https://github.com/apache/maka/blob/main/runtime-kernel.ts)—passes this policy to `applyRuntimeEventContextBudget` at turn start.

## Why Mako's Budget Is Advisory, Not Absolute

As noted in comments at lines 68-71 of `ContextBudgetPolicy`, **the provider remains the ultimate authority** on token limits. Mako's estimates approximate actual tokenization; providers may reject requests that appear within budget. When this occurs, the runtime can apply **additional trimming** or **fallback to smaller context windows**.

This design acknowledges a fundamental reality: different providers tokenize differently, and runtime-side budgets must balance **proactive optimization** against **defensive flexibility**.

## Observability Through Diagnostics

Every budget application produces a `ContextBudgetDiagnostic` (see `minimalContextBudgetDiagnostic` in [`context-budget.ts`](https://github.com/apache/maka/blob/main/context-budget.ts)). These records enable operators to tune policies based on empirical data:

```typescript
// Example diagnostic output
{
  enabled: true,
  policyName: "production-budget",
  estimatedTokensBefore: 15420,
  estimatedTokensAfter: 8934,
  keptTurns: 12,
  droppedTurns: 8,
  keptEvents: 127,
  droppedEvents: 73
}

```

Monitoring these diagnostics reveals whether budgets are too aggressive (quality loss) or too conservative (unnecessary cost).

## Practical Implementation Example

Complete workflow integrating budget application with provider communication:

```typescript
import { 
  applyRuntimeEventContextBudget, 
  ContextBudgetPolicy 
} from '@maka/runtime';

// 1. Define budget constraints
const budgetPolicy: ContextBudgetPolicy = {
  name: "tight-budget",
  charsPerToken: 4,
  activeToolResultPrune: { enabled: true, maxTokens: 2048 },
  historyCompact: { enabled: true, maxEvents: 300 },
};

// 2. Apply to current event stream (typically inside backend adapter)
const budgeted = applyRuntimeEventContextBudget(currentEvents, budgetPolicy);

if (budgeted) {
  // 3. Project events to provider-specific messages
  const providerMessages = buildProviderMessages(budgeted.events);
  
  // 4. Log diagnostics for monitoring
  console.info("Context budget applied:", budgeted.diagnostic);
  
  // 5. Execute LLM call with constrained payload
  const response = await llmClient.complete(providerMessages);
}

```

## Key Source Files and Responsibilities

| File | Purpose | Critical Functions/Types |
|------|---------|--------------------------|
| [`packages/runtime/src/context-budget.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/context-budget.ts) | Core budgeting logic | `applyRuntimeEventContextBudget`, `applyRuntimeEventHistoryCompactNarrow`, `ContextBudgetPolicy`, `ContextBudgetDiagnostic` |
| [`packages/runtime/src/context-budget-helpers.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/context-budget-helpers.ts) | Estimation utilities | `estimateTokens`, `utf8ByteLength`, `estimateRuntimeEventsTokens` |
| [`packages/runtime/src/ai-sdk-backend.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/ai-sdk-backend.ts) | Provider message construction | Converts budgeted `RuntimeEvent[]` to provider-native format |
| [`packages/runtime/src/runtime-kernel.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/runtime-kernel.ts) | Turn orchestration | Entry point invoking budget application |
| [`packages/runtime/src/__tests__/context-budget.test.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/__tests__/context-budget.test.ts) | Behavioral verification | Unit tests for compaction, pruning, estimation accuracy |

## Summary

- **Mako manages context and token budgets through pure-functional projection** of the immutable Runtime Event Log, never mutating persisted state.

- **Three pipeline stages**—token estimation, stale tool result pruning, and history compaction—progressively reduce payload size while preserving semantic correctness.

- **`applyRuntimeEventContextBudget`** in [`packages/runtime/src/context-budget.ts`](https://github.com/apache/maka/blob/main/packages/runtime/src/context-budget.ts) orchestrates the complete workflow, returning both constrained events and diagnostic metadata.

- **Budget policies are client-configurable** via `ContextBudgetPolicy`, specifying character-per-token ratios, pruning thresholds, and compaction limits.

- **Provider authority remains ultimate**; Mako's estimates optimize proactively but cannot guarantee acceptance, requiring runtime fallback strategies.

- **Comprehensive diagnostics** enable data-driven policy tuning through `ContextBudgetDiagnostic` records logged on every request.

## Frequently Asked Questions

### How does Mako estimate tokens without calling an LLM tokenizer?

Mako uses a **configurable character-to-token ratio** (default 4 characters/token) implemented in `estimateTokens` within [`context-budget-helpers.ts`](https://github.com/apache/maka/blob/main/context-budget-helpers.ts). This approximation avoids tokenizer latency while providing sufficient accuracy for budget decisions. The `charsPerToken` policy field allows adjustment for different model families with varying tokenization densities.

### Can stale tool result pruning cause information loss during replay?

**No.** Stale tool result pruning is a **pure projection** applied only to the temporary request view. The original `RuntimeEvent` containing the full tool result remains in the persisted log. According to the architecture documentation, this guarantees that replay and recovery operations see complete, unmodified history regardless of budget constraints applied to active requests.

### What happens when a provider rejects a request that Mako believed was within budget?

The runtime implements **defensive fallback strategies**. Since provider tokenization may differ from Mako's estimates (noted in `ContextBudgetPolicy` comments L68-71), the system can apply additional compaction, reduce `maxEvents` thresholds, or switch to smaller context windows. These adjustments occur at the `AiSdkBackend` layer before retrying the request.

### How can operators monitor whether their budget policies are effective?

Every budget application produces a `ContextBudgetDiagnostic` object containing **beforE/after token estimates**, **kept/dropped turn counts**, and **event retention metrics**. Operators should aggregate these diagnostics—particularly `estimatedTokensBefore`, `estimatedTokensAfter`, and `droppedTurns`—to identify policies that are too aggressive (excessive turn loss) or too conservative (insufficient savings).