# Turso Database Recovery Process: Automatic Crash Recovery After Power Failure

> Learn how Turso Database automatically recovers from crashes and power failures. Discover how WAL, MVCC logs, and checksum validation ensure data consistency and fast restoration.

- Repository: [Turso Database/turso](https://github.com/tursodatabase/turso)
- Tags: how-to-guide
- Published: 2026-06-23

---

**Turso automatically restores database consistency after a crash or power failure by replaying the Write-Ahead Log (WAL) and MVCC logical log, validating headers with checksums, and atomically back-filling pages before truncating the WAL.**

The Turso database engine (`tursodatabase/turso`) implements a robust **recovery process for Turso databases** that ensures ACID durability even after sudden power loss or operating system crashes. Built on a SQLite-compatible storage layer, the system leverages a three-phase protocol that detects incomplete writes and reconstructs committed transaction states without manual intervention.

## How Crash Recovery Works in Turso

When an operating system crashes or power is cut, Turso may be left with a partially-written Write-Ahead Log (WAL) and an out-of-date on-disk B-tree. The engine resolves this by replaying the WAL through the MVCC logical log the next time the database is opened. This recovery flow is built into the core storage layer and orchestrated by the `OpenSharedWal::poll` state machine defined in [`core/storage/wal.rs`](https://github.com/tursodatabase/turso/blob/main/core/storage/wal.rs).

## The Three Phases of Recovery

Turso's recovery protocol consists of tightly-coupled phases that execute sequentially when `Database::open` is invoked.

### 1. WAL Header Validation and Restart

The engine begins by reading the WAL header (`WAL_HEADER_SIZE` bytes) to verify the magic number, page size, and checksum. If the header is missing, corrupted, or indicates an interrupted checkpoint, the engine executes a restart sequence.

In [`core/storage/wal.rs`](https://github.com/tursodatabase/turso/blob/main/core/storage/wal.rs), the `prepare_wal_header` function performs initial validation, while `begin_restart` rewrites the header to a clean state and clears stale read-marks. This follows the SQLite-compatible recovery protocol, ensuring the log is in a known good state before replay begins.

### 2. MVCC Logical Log Replay

Turso uses a separate **MVCC logical log** (an on-disk file storing MVCC frames) to replay operations committed to the WAL but not yet back-filled into the main database file. The recovery logic iterates over each frame from the last known `max_frame` to the current file size.

For every frame encountered, the engine:
- Verifies the CRC checksum to detect corruption
- Deserializes the payload
- Applies the page updates to the in-memory B-tree via `LogicalLog::apply_frame`
- Updates the running CRC state to resume subsequent writes

This logic resides in [`core/mvcc/persistent_storage/logical_log.rs`](https://github.com/tursodatabase/turso/blob/main/core/mvcc/persistent_storage/logical_log.rs) within the `recover` and `apply_frame` functions.

### 3. Checkpoint and Cleanup

Once logical-log replay completes, Turso performs an implicit checkpoint. The engine back-fills newly-replayed pages into the database file and updates the WAL's `max_frame` and `nbackfills` counters.

If a `TRUNCATE` checkpoint was pending, `Wal::truncate_wal` (also in [`core/storage/wal.rs`](https://github.com/tursodatabase/turso/blob/main/core/storage/wal.rs)) shrinks the WAL to zero bytes. Critically, truncation occurs **only after** the database file has been synced to disk, preserving durability guarantees.

## Step-by-Step Crash Recovery Walkthrough

The following sequence occurs when reopening a database after an unexpected shutdown:

1. **Crash occurs** – The process terminates abruptly, potentially leaving the WAL with a partially-written frame or unfinished checkpoint.
2. **Database reopened** – `Database::open` constructs a WAL implementation (`SharedWalCoordination` → `InProcessWalCoordination` or `ShmWalCoordination` for cross-process use).
3. **Header preparation** – `prepare_wal_header` validates the existing header or initializes a fresh one if uninitialized.
4. **Poll execution** – `OpenSharedWal::poll` runs iteratively:
   - Reads the WAL header and checks magic numbers and checksums
   - Invokes `begin_restart` if the header indicates a restart is required
   - Scans WAL frames, calling `LogicalLog::apply_frame` for each
5. **Logical-log replay** – Frame payloads apply to the B-tree while `LogicalLog::running_crc` updates.
6. **Implicit checkpoint** – The engine back-fills pages via `Wal::checkpoint` and potentially calls `Wal::truncate_wal`.
7. **Ready state** – The `Database` object returns to the caller with a fully-recovered, consistent state.

## Automatic Recovery Example

No special API calls are required to trigger recovery. Simply reopening the database executes the full recovery protocol:

```rust
use turso::Database;

// Open database (creates WAL if missing)
let mut db = Database::open("my.db", Default::default())?;

// Write transaction - WAL flushes but we don't checkpoint
db.execute("INSERT INTO demo (id, txt) VALUES (1, 'first')")?;

// Simulate crash by exiting without graceful shutdown
std::process::exit(0);   // Power failure simulation

// After process restart
let db = Database::open("my.db", Default::default())?;

// Committed data is visible despite the crash
let row: (i64, String) = db.query_row("SELECT id, txt FROM demo")?;
assert_eq!(row, (1, "first".to_string()));

```

## Manual Recovery Control for Testing

For integration testing or custom tooling, you can drive the recovery loop manually using the low-level API:

```rust
use turso::storage::wal::OpenSharedWal;

// Assume `wal` is a `Box<dyn Wal>` obtained from `Database::open`
let mut poll = OpenSharedWal::new(&wal);
loop {
    match poll.poll()? {
        turso::storage::wal::PollResult::Done => break,
        turso::storage::wal::PollResult::Pending => continue,
    }
}

```

This pattern allows tests to inspect intermediate states or verify idempotency—that running recovery twice produces the same result.

## Durability Guarantees and Verification

Turso guarantees **Atomicity, Consistency, and Durability** after sudden failure through:

- **Checksum validation** detecting incomplete WAL writes via header CRCs
- **Sequential replay** of every committed WAL frame through the MVCC logical log
- **Safe truncation** of the WAL only after the database file sync completes

The correctness of this recovery path is continuously verified in:

- **[`tests/integration/query_processing/test_transactions.rs`](https://github.com/tursodatabase/turso/blob/main/tests/integration/query_processing/test_transactions.rs)** – Simulates crashes by closing the DB without checkpointing, then re-opening to assert committed rows remain visible
- **[`testing/simulator/runner/memory/mvcc_recovery.rs`](https://github.com/tursodatabase/turso/blob/main/testing/simulator/runner/memory/mvcc_recovery.rs)** – Stress-tests recovery idempotency by running the loop repeatedly

## Summary

- Turso's **recovery process for Turso databases** runs automatically on `Database::open`, requiring no manual intervention.
- The protocol follows three phases: **WAL header validation** (`prepare_wal_header`), **MVCC logical log replay** (`LogicalLog::apply_frame`), and **checkpoint truncation** (`truncate_wal`).
- **Checksum validation** at the header and frame level detects corruption from partial writes.
- Durability is ensured by syncing the database file before truncating the WAL.
- Recovery is **idempotent** and safe to run multiple times, as verified by the simulator tests.

## Frequently Asked Questions

### Is the Turso database recovery process automatic?

Yes. When you call `Database::open` after a crash, the engine automatically creates an `OpenSharedWal` instance and drives the recovery state machine via repeated `poll` calls until completion. No separate recovery command or configuration is required.

### What happens if the WAL header is corrupted?

If `prepare_wal_header` detects a missing magic number, invalid page size, or checksum mismatch, the engine invokes `begin_restart` to reset the WAL header to a clean, initialized state. This follows SQLite-compatible recovery semantics and allows the database to start fresh while preserving back-filled data in the main database file.

### How does Turso ensure data durability during recovery?

Durability is guaranteed through a specific ordering of operations: the engine writes and syncs all back-filled pages to the main database file before calling `truncate_wal` to delete WAL segments. Additionally, every WAL frame and logical log entry includes CRC checksums that `LogicalLog::apply_frame` validates before applying updates.

### Can I run recovery manually for testing purposes?

Yes. You can instantiate `OpenSharedWal` directly and manually drive the `poll` loop, as shown in the code example above. This is useful for integration tests that need to verify specific intermediate states or confirm that the recovery process is idempotent—running it twice on the same WAL produces identical results.