AgentRun Lifecycle and Crash Recovery in Apache Maka: A Complete Technical Guide
The Apache Maka AgentRun lifecycle consists of three phases—creation, progress, and termination—where all events are persisted to an immutable append-only log, enabling crash recovery through deterministic event replay on restart.
Apache Maka's execution model is built around a single authoritative structure: the AgentRun. An AgentRun represents the complete, ordered stream of events that records everything occurring during an active session. This stream serves as the sole source of truth for both progress tracking and crash recovery, making the AgentRun lifecycle and crash recovery mechanism central to understanding how Maka achieves reliable, durable execution.
What Is an AgentRun?
An AgentRun is an immutable, append-only sequence of events that captures the entire history of a session. From the moment a session starts through every turn of interaction to final termination, every observable action becomes a persisted event. This design eliminates data loss during unexpected failures and enables complete state reconstruction.
The core data structures defining an AgentRun are located in packages/core/src/agent-run.ts, which declares:
AgentRunHeader— contains immutable identifiers (sessionId,runId,turnId,invocationId)AgentRunEvent— represents individual actions (tool calls, model responses, system events)AgentRunEventType— enumerates event categoriesAgentRunProjectionKey— enables efficient querying and indexing
Phase 1: AgentRun Creation
When a SessionManager initiates a new run, it emits an AgentRunHeader that establishes the run's identity. This header is immediately persisted to durable storage before any execution begins.
The creation process in packages/storage/src/agent-run-store.ts ensures:
- Unique identification through the quadruple (
sessionId,runId,turnId,invocationId) - Atomic header persistence to SQLite via
createSqliteAgentRunStore - Prevention of duplicate run creation for the same session context
import { createSqliteAgentRunStore } from '@maka/storage';
// Initialize durable storage for the workspace
const agentRunStore = createSqliteAgentRunStore(workspaceRoot);
// Start a new run with immutable identifiers
const header = await agentRunStore.createRun({
sessionId: 'sess-123',
runId: 'run-001',
turnId: 0,
invocationId: 'inv-xyz',
});
The SQLite schema in packages/storage/src/sqlite-usage-schema.ts explicitly documents this design principle: "The AgentRun sequence is the projection's sole progress authority" (lines 60-61).
Phase 2: Event Progression and Persistence
During active execution, every significant action appends an AgentRunEvent to the store. The DurableAgentRunStore interface and its SqliteAgentRunStore implementation handle this in packages/storage/src/agent-run-store.ts.
Events that trigger persistence include:
- Tool requests and responses
- Model calls and outputs
- System state transitions
- Error conditions
// Append a tool request event
await agentRunStore.appendEvent({
...header,
type: 'TOOL_REQUEST',
payload: { tool: 'search', query: 'latest Apache releases' },
});
// Append a model response
await agentRunStore.appendEvent({
...header,
type: 'MODEL_RESPONSE',
payload: { content: 'Apache Maka is an open-source...', tokens: 127 },
});
The append method validates event identity against the header through decodeAgentRunEvent in packages/storage/src/execution-record-codec.ts (lines 71-81). This validation throws on mismatch, ensuring consistency between headers and events.
Phase 3: Termination and Immutability
A run terminates when a terminal event—AgentRunEventType.DONE or AgentRunEventType.ERROR—is recorded. Post-termination, the store enforces immutability: the append method rejects any further write attempts for that run.
This immutability guarantee enables safe recovery semantics and auditability. Once terminated, a run's history becomes a permanent, tamper-evident record of execution.
How Crash Recovery Works in Apache Maka
Maka's crash recovery mechanism leverages the durable AgentRun log to reconstruct state after any failure—process exit, power loss, or host crash. Since every side-effecting action is persisted before execution, no work is lost.
Recovery proceeds through three deterministic steps:
Step 1: Detect Incomplete Runs
On startup, the RuntimeHost queries for runs lacking terminal events via listSessionRunsForRecovery:
// Find the most recent incomplete run for recovery
const runs = await agentRunStore.listSessionRunsForRecovery(sessionId);
if (!runs.length) return; // Clean state, nothing to recover
const runHeader = runs[0];
This method is defined in packages/storage/src/agent-run-store.ts (lines 232-235) and returns headers ordered by recency, prioritizing the most recent interrupted work.
Step 2: Replay Events Deterministically
The host retrieves all events via readEventsForRecovery and re-executes them in order to rebuild in-memory state:
async function recoverRun(sessionId: string) {
const runs = await agentRunStore.listSessionRunsForRecovery(sessionId);
if (!runs.length) return;
const runHeader = runs[0];
const events = await agentRunStore.readEventsForRecovery(
runHeader.sessionId,
runHeader.runId,
);
for (const ev of events) {
// Deterministic rehydration: re-issue pending calls, restore context
await handleEvent(ev);
}
}
The Runtime Resume architecture document at docs/architecture/runtime-resume-architecture.md specifies this replay algorithm and its safety properties.
Step 3: Resume Execution
After state reconstruction, execution continues from the last persisted event:
- If the final event was a pending tool request, the host re-issues the tool call
- If it was a model response, the host proceeds to the next turn
- If it was an incomplete operation, the operation restarts with idempotency guarantees
Safety Guarantees of the AgentRun Design
| Property | Implementation | Verification |
|---|---|---|
| Durability | SQLite WAL mode with synchronous writes | Events persisted before side-effects execute |
| Consistency | Header-event identity validation | decodeAgentRunEvent throws on identifier mismatch |
| Idempotence | Deterministic event replay | Same input sequence yields identical state |
| Isolation | Single active run per session | AgentRunStore enforces mutual exclusion |
The append-only, tamper-evident design in packages/storage/src/execution-record-codec.ts ensures that any data corruption is detected during recovery, preventing silent failures.
Real-World Usage in Maka's Reference Implementation
The scripts/computer-use/real-model.mjs script demonstrates production usage of the AgentRun lifecycle:
// From real-model.mjs — production pattern for AgentRun store creation
import { createSqliteAgentRunStore } from '@maka/storage';
const workspaceRoot = process.env.MAKA_WORKSPACE || './.maka';
const agentRunStore = createSqliteAgentRunStore(workspaceRoot);
// Store passed to model runtime for automatic lifecycle management
const runtime = createModelRuntime({ agentRunStore, sessionConfig });
This pattern (lines 32-33 in the source) shows how the store integrates with higher-level runtime components without requiring manual event management.
Comparison: AgentRun vs. Traditional Checkpointing
| Approach | State Capture | Recovery Granularity | Complexity |
|---|---|---|---|
| AgentRun event log | Every observable action | Per-event | Low — deterministic replay |
| Periodic checkpointing | Snapshots at intervals | Last checkpoint | High — state serialization, missed events between checkpoints |
| Transactional memory | Memory pages | Page-level | High — requires hardware/OS support |
The event log approach trades storage size for simplicity and correctness: recovery is always accurate to the last completed event, with no ambiguity about partial operations.
Summary
- AgentRun is Apache Maka's append-only event log that serves as the single source of truth for execution state
- The lifecycle spans creation (header emission), progression (event appending), and termination (immutable completion)
- Crash recovery detects incomplete runs via
listSessionRunsForRecovery, replays events throughreadEventsForRecovery, and resumes deterministically - Safety guarantees include durability (SQLite persistence), consistency (header validation), idempotence (deterministic replay), and isolation (single active run)
- Core implementation files:
packages/core/src/agent-run.ts(types),packages/storage/src/agent-run-store.ts(persistence),packages/storage/src/execution-record-codec.ts(validation)
Frequently Asked Questions
How does Apache Maka ensure no data loss during a crash?
All events are written to SQLite using WAL mode with synchronous commits before any side-effect executes. The SqliteAgentRunStore implementation in packages/storage/src/agent-run-store.ts guarantees that once an appendEvent call returns, the event is durable. Recovery simply replays these persisted events on restart.
What happens if event replay fails during recovery?
The decodeAgentRunEvent validator in packages/storage/src/execution-record-codec.ts (lines 71-81) checks that each event's identifiers match its header. Any corruption or tampering triggers an exception, halting recovery rather than proceeding with inconsistent state. Operators can then inspect the event log for manual intervention.
Can multiple runs be active for the same session simultaneously?
No — the AgentRun design enforces isolation at the session level. The store implementation prevents concurrent run creation, and the lifecycle ensures only one run transitions from creation to termination per session. This prevents log corruption from interleaved events.
Is the recovery process idempotent if restarted multiple times?
Yes. Because event replay is deterministic and the run state is reconstructed from scratch each time, repeated recovery attempts yield identical results. The RuntimeHost detects whether a run has already recovered through terminal event checks, avoiding duplicate execution of completed work.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" Maintain an open-source project? Get it listed too →