Runtime Event Log in Apache Maka: Immutable Ledger and Projection Architecture

Apache Maka implements a log-first architecture where every user interaction, model output, and tool result is stored as an immutable RuntimeEvent, making the Runtime Event Log the single source of truth while context budgeting and history compaction operate as lossy projections layered on top.

The Runtime Event Log in Apache Maka serves as the canonical ledger for all runtime activity. Unlike traditional systems that mutate history to fit context windows, Maka preserves every semantic fact in an append-only sequence, enabling full auditability, deterministic replay, and flexible recovery. Higher-level features materialize views of this log rather than modifying it, according to the design specified in docs/architecture/llm-compaction-events-log-projection-draft.md.

The Three-Layer Architecture

The architecture separates concerns into distinct layers with different durability guarantees:

• Runtime Events Log: Stores all raw semantic facts produced by users, models, tools, and the Runtime itself. This is the canonical source of truth that never loses detail.
• History Compact Checkpoint: A lossy projection containing either a V2 text summary or V3 provider-native state plus coverage metadata. This is a durable derivation of the log, not the source.
• Provider Request Messages: The concrete payload sent to the LLM, generated on-the-fly by materializing checkpoints with raw event tails.

The relationship follows this functional pattern:

Canonical history = RuntimeEvents[0..n]

Compact checkpoint = Project(
    RuntimeEvents[0..k],
    compaction policy,
    summarizer
)

Next model context = Materialize(
    compact checkpoint,
    RuntimeEvents[k+1..n],
    provider capabilities,
    context budget
)

Immutable Ledger Design

Every RuntimeEvent appends to a persistent ledger without in-place modification. The system distinguishes between RuntimeEvent (raw facts) and AgentRunEvent (which stores the durable history_compact_checkpoint_recorded events). No compaction ever rewrites or deletes existing entries; instead, the system appends checkpoint records to the log itself, preserving the complete history of both data and compaction decisions.

Projection-First Compaction Pipeline

When accumulated events exceed the model’s context window, Maka executes a five-step projection pipeline implemented in packages/runtime/src/ai-sdk-backend.ts (specifically within buildPriorMessages()):

1. High-water selection: The system calculates the largest safe prefix of events that fits within the token budget using packages/runtime/src/context-budget.ts.
2. Summarizer invocation: For V2 checkpoints, packages/runtime/src/history-compact-summarizer.ts generates a text summary. For V3, packages/runtime/src/openai-codex-history-compactor.ts produces provider-native state.
3. Checkpoint validation: The system verifies coverage metadata—including event count, turn count, runId/turnId/runtimeEventId boundaries, and SHA-256 digest—against the source prefix.
4. Durable append: Upon validation, the system atomically persists a history_compact_checkpoint_recorded event via packages/runtime/src/agent-run.ts, ensuring the checkpoint is only used after durable storage succeeds.
5. Materialization: The next provider request consists of the checkpoint (as a synthetic RuntimeEvent for V2 or custom part for V3) plus the raw tail of uncovered events.

Rolling Checkpoints and Incremental Updates

Rather than summarizing the entire history repeatedly, Maka builds incremental checkpoints to minimize LLM load:

Checkpoint N   : summary = S(events[0..k])
Checkpoint N+1 : summary = S(Checkpoint N.summary, events[k+1..m])

Both V2 and V3 checkpoint schemas—defined in packages/runtime/src/history-compact-checkpoint.ts—carry identical coverage metadata, allowing fast prefix-only updates without resending already-summarized events to the LLM. This rolling approach maintains the invariant that newer events remain raw while older prefixes transition into summarized form.

Recovery and Replay Mechanics

Recovery logic in packages/runtime/src/history-compact-ledger.ts scans the canonical AgentRun ledger to locate the newest valid checkpoint when the bounded cache is missing or corrupted. Replay reconstructs the provider request as checkpoint + uncovered tail + current turn. If a checkpoint no longer satisfies the current policy—such as when token budgets change—the system falls back to using the raw event tail, ensuring continuity without data loss.

Architectural Invariants and Guarantees

The Runtime Event Log architecture enforces six strict invariants:

• Source immutability: Original events never change.
• Coverage verification: Each checkpoint must cryptographically prove it covers an ordered prefix via event counts and SHA-256 digests.
• Durable ordering: Checkpoints are only utilized after atomic durable append succeeds.
• Monotonic high-water: Checkpoints normally advance forward; equal-coverage rewrites must be explicit successors.
• Policy re-evaluation: Previously accepted checkpoints are re-verified against current context budgets before every request.
• Raw recent tail: The newest events are always preserved in raw form to guarantee up-to-date facts.

Working with the Runtime Event Log

Developers interact with the log through specific APIs exposed by the runtime kernel and storage layers.

Triggering Manual Compaction

To manually request compaction (fails if a turn is active):

import { RuntimeKernel } from '@maka/runtime';
import { Session } from '@maka/sdk';

await session.run(async (kernel: RuntimeKernel) => {
  // Creates a Turn/Run with a Compact command
  await kernel.compactSession();
});

This executes the same high-water selection and validation pipeline described above.

Inspecting Checkpoints Programmatically

Access the latest checkpoint metadata via the storage layer:

import { AgentRunStore } from '@maka/storage';
import { HistoryCompactCheckpoint } from '@maka/runtime';

async function getLatestCheckpoint(sessionId: string) {
  const store = new AgentRunStore();
  const checkpoint = await store.getLatestHistoryCompactCheckpoint(sessionId);
  return checkpoint as HistoryCompactCheckpoint; 
  // Contains coverage, digest, V2/V3 payload
}

Building Provider Requests

The runtime internally materializes context through the AI SDK backend:

import { AiSdkBackend } from '@maka/runtime';

async function buildProviderMessage(sessionId: string) {
  const backend = new AiSdkBackend();
  const priorMessages = await backend.buildPriorMessages(sessionId);
  // Contains checkpoint (if any) + raw tail
  return priorMessages;
}

Key Implementation Files

The log-first architecture is implemented across these specific modules:

• packages/runtime/src/context-budget.ts: Calculates high-water marks and validates replay policies.
• packages/runtime/src/history-compact-checkpoint.ts: Defines V2/V3 schemas and digest verification.
• packages/runtime/src/ai-sdk-backend.ts: Orchestrates the prior-message pipeline and manual compaction.
• packages/runtime/src/agent-run.ts: Persists history_compact_checkpoint_recorded events.
• packages/runtime/src/history-compact-ledger.ts: Manages bounded-projection cache and recovery scanning.
• packages/storage/src/agent-run-store.ts: Guarantees atomic ordering between event append and checkpoint updates.

Summary

• The Runtime Event Log is an immutable, append-only ledger serving as the single source of truth for all runtime activity.
• History compaction operates as a projection (V2 text or V3 provider-native state) layered on top of the canonical log, never modifying original events.
• Rolling checkpoints enable incremental summarization, reducing LLM load while maintaining coverage metadata for verification.
• Recovery scans the AgentRun ledger to rebuild state from the newest valid checkpoint, with fallback to raw events if policies change.
• All compaction workflows enforce strict invariants including SHA-256 digest verification, durable ordering, and monotonic high-water advancement.

Frequently Asked Questions

How does Apache Maka handle context window limitations without losing data?

Apache Maka projects a prefix of the Runtime Event Log into a lossy checkpoint (V2 summary or V3 provider state) while preserving the original events immutably. When building provider requests, it materializes the checkpoint plus the raw tail of recent events, ensuring the full history remains available for audit even as the active context stays within token limits.

What is the difference between V2 and V3 checkpoints in the Runtime Event Log?

V2 checkpoints are text summaries generated by the LLM itself via packages/runtime/src/history-compact-summarizer.ts, while V3 checkpoints are provider-native state objects created by specific implementations like packages/runtime/src/openai-codex-history-compactor.ts. Both formats carry identical coverage metadata (event counts, digests, and boundary IDs) and follow the same validation and durable append workflow.

How does the system recover from checkpoint corruption or cache misses?

The packages/runtime/src/history-compact-ledger.ts module implements recovery logic that scans the canonical AgentRun ledger for the newest valid history_compact_checkpoint_recorded event. If the checkpoint no longer meets current policy requirements—such as changed token budgets—the system falls back to materializing the raw event tail, ensuring continuous operation without data loss.

Can developers manually trigger history compaction in Apache Maka?

Yes, developers can invoke manual compaction through the RuntimeKernel.compactSession() API, available when a turn is not active. This triggers the standard five-step projection pipeline (high-water selection, summarizer invocation, validation, durable append, and materialization) through the same code path used for automatic compaction in packages/runtime/src/ai-sdk-backend.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}