How Mako Reconstructs Execution State After a Crash Using SQLite and the Event Log

Mako guarantees deterministic crash recovery by rebuilding sessions from an immutable, append-only RuntimeEvent log stored in a SQLite database with WAL durability.

Mako's crash recovery system ensures that interrupted AI agent sessions can resume safely from exactly where they left off. At the heart of this capability lies a carefully designed persistence layer combining SQLite's atomic transactions with a structured event log that records every turn, tool call, and response.

Why SQLite Powers Mako's Durability Guarantees

Mako selected SQLite for its event store because three properties align perfectly with crash recovery requirements:

• Atomicity — SQLite's Write-Ahead Log (WAL) mode guarantees each RuntimeEvent row is either fully persisted or not present at all, eliminating torn writes
• Consistency — The committedRuntimeEventHighWater column tracks the last safely stored row number, creating a clear boundary between trustworthy history and incomplete work
• Fast replay — Sequential scans of the append-only RuntimeEvent table avoid costly joins, enabling efficient projection of execution state

The SQLite database typically resides at ~/.local/share/opencode/opencode.db and operates exclusively in WAL mode to maximize durability.

The Three-Phase Recovery Flow

Mako's crash reconstruction follows a strict protocol implemented in packages/runtime-host/src/server/root-turn-coordinator.ts. When a host process restarts after failure, it executes three coordinated steps:

Phase 1: Persist a Committed Event Prefix

Every RuntimeEvent writes inside a SQLite WAL transaction that is fully fsync-ed before proceeding. If the host process dies, the database contains only a committed prefix of events — no partial records survive. This append-only discipline ensures the event log remains immutable and auditable.

Phase 2: Reopen and Locate the High-Water Mark

On startup, the new host process reopens the same SQLite file and reads committedRuntimeEventHighWater from the SQLiteRuntimeStore. This integer marks the boundary separating reliable history from any in-flight work that may have been lost.

import { SQLiteRuntimeStore } from './packages/storage/src/sqlite-runtime-store';

const store = new SQLiteRuntimeStore({ 
  path: '/home/user/.local/share/opencode/opencode.db' 
});
await store.open();

const highWater = await store.committedRuntimeEventHighWater(); // e.g. 12345
const events = await store.readEvents({ start: 0, end: highWater });
console.log(`Recovered ${events.length} committed events`);
await store.close();

Phase 3: Project and Verify the Safe-Boundary Continuation

The RootTurnCoordinator calls planAuthoritativeSafeBoundaryContinuation to transform the recovered prefix into a ResumePlan. This projection step implements the Phase 0 crash contract documented in docs/architecture/runtime-resume-phase0-crash-contract.md.

// Core reconstruction logic from root-turn-coordinator.ts
private async reconstructAdmittedContinuation(
  admission: RootTurnAdmission,
): Promise<ReconstructedContinuation> {
  const execution = admission.execution;
  if (execution.kind !== 'safe_boundary_continuation') {
    throw new RuntimeMessageAuthorityInvariantError(
      'Only safe-boundary continuation admission can reconstruct a continuation',
    );
  }

  // 1️⃣ Project the committed RuntimeEvent prefix into a resumption plan
  const plan = await this.manager.planAuthoritativeSafeBoundaryContinuation(
    admission.sessionId,
    {
      sourceRunId: execution.sourceRunId,
      expectedRuntimeEventHighWater: execution.sourceRuntimeEventHighWater,
    },
  );
  const projection = projectTurnResumePlan(admission.sessionId, plan);

  // 2️⃣ Verify all source proofs match the original admission
  if (projection.disposition === 'parked') {
    return { disposition: 'parked', plan };
  }
  
  const planned = requirePlannedContinuation(plan);
  if (
    planned.sourceInvocationId !== execution.sourceInvocationId ||
    planned.sourceRunId !== execution.sourceRunId ||
    planned.sourceTurnId !== execution.sourceTurnId ||
    planned.sourceRuntimeEventHighWater !== execution.sourceRuntimeEventHighWater ||
    planned.boundary?.manifestDigest !== execution.boundaryDigest ||
    planned.providerReplayDigest !== execution.providerReplayDigest
  ) {
    throw new RuntimeMessageAuthorityInvariantError(
      'Safe-boundary continuation source proof changed after admission',
    );
  }

  // 3️⃣ Final safety digest verification
  if (continuationSafetyDigest(planned) !== execution.safetyDigest) {
    return {
      disposition: 'parked',
      plan: parkedTurnResumePlan(admission.sessionId, 'safety_check_failed'),
    };
  }

  // 4️⃣ Construct the validated continuation for resumption
  return {
    disposition: 'ready',
    continuation: {
      ...planned,
      invocationId: execution.targetInvocationId,
      runId: admission.runId,
      turnId: admission.turnId,
      claimId: execution.claimId,
    },
  };
}

How the Event Log Drives Reconstruction Decisions

The RuntimeEventStore records every significant operation:

Event Type	Purpose
Model turns	LLM requests and responses
Tool calls	Invocation of external capabilities
Tool responses	Results returned from tools
Terminal events	Completion markers that define safe boundaries

After a crash, the host projects this log through the plan*Continuation APIs to determine if the recovered prefix permits safe resumption. The Phase 0 crash contract defines exact fail-points and expected outcomes for each possible prefix state:

• before_function_call — Tool requested but not yet invoked; safe to replay
• after_function_response — Tool completed; boundary digest must verify
• after_terminal_event — Turn complete; yields full reconstruction

If the prefix ends at a terminal event, reconstruction yields a complete turn. Otherwise, the system either parks the continuation (awaiting user intervention) or reports a safe replay with no pending tool operation.

Practical Recovery Example

The following pattern demonstrates complete crash recovery from a host restart:

import { RootTurnCoordinator } from './packages/runtime-host/src/server/root-turn-coordinator';

// Initialize coordinator with dependency injection for stores
const coordinator = new RootTurnCoordinator(/* runtimeDependencies */);

// Recover session "sess-123" after crash during turn 42
const admission = await coordinator.stores.agentRunStore.readRootTurnAdmission(
  'sess-123',
  42, // turnId
);

if (!admission) {
  throw new Error('No admission record found for crashed turn');
}

const recon = await coordinator.reconstructAdmittedContinuation(admission);

switch (recon.disposition) {
  case 'ready':
    // All safety checks passed — resume execution immediately
    await coordinator.resumeTurn(recon.continuation);
    console.log('Turn resumed successfully');
    break;
    
  case 'parked':
    // Safety verification failed or incomplete prefix
    console.warn('Turn parked — requires manual review:', recon.plan);
    // Render blocked state in UI
    break;
    
  default:
    // Exhaustive case handling for type safety
    const _exhaustive: never = recon;
}

Key Source Files Supporting Crash Recovery

File	Responsibility
packages/runtime-host/src/server/root-turn-coordinator.ts	Core reconstructAdmittedContinuation algorithm that validates and rebuilds continuations from safe-boundary admissions
packages/storage/src/sqlite-runtime-store.ts	Low-level SQLite wrapper implementing WAL persistence, high-water mark tracking, and sequential event reads
docs/architecture/runtime-resume-phase0-crash-contract.md	Formal specification of committed prefixes, fail-points, and expected reconstruction outcomes
packages/storage/src/workspace-version-authority-internal.ts	Transactional contract guaranteeing atomic event log updates across SQLite transactions
packages/storage/src/sqlite-runtime-schema.ts	Immutable RuntimeEvent row definitions constituting the event log

Summary

• SQLite WAL transactions guarantee atomic, durable writes of every RuntimeEvent before process continuation
• committedRuntimeEventHighWater provides a deterministic boundary between committed history and lost in-flight work
• reconstructAdmittedContinuation projects the safe prefix, verifies integrity digests, and produces a RuntimeContinuation or parked state
• The Phase 0 crash contract formalizes expected behavior across all possible recovery scenarios
• Verification failures result in parking rather than unsafe execution, prioritizing correctness over availability

Frequently Asked Questions

What happens if the safety digest verification fails during reconstruction?

The reconstruction returns disposition: 'parked' with a plan indicating safety_check_failed. This conservative behavior prevents execution from continuing when integrity cannot be proven, requiring manual operator review rather than risking incorrect state.

How does Mako prevent torn writes in the event log?

Every RuntimeEvent writes inside a SQLite transaction with full fsync synchronization before the process proceeds. SQLite's WAL mode ensures that even if the OS crashes mid-write, the database on disk contains only complete, committed rows — no partial event data can survive.

Can multiple host processes access the same SQLite database simultaneously?

Mako's architecture assumes single-writer access to the event log per session. The SQLite database supports concurrent readers, but write coordination happens through the single RootTurnCoordinator instance responsible for a given session.

Where is the event log physically stored on disk?

By default, Mako stores the SQLite database at ~/.local/share/opencode/opencode.db. This path is configurable through the SQLiteRuntimeStore constructor's path option, as shown in the debugging example above.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}