Checkpointing and WAL Mode in Turso: A Complete Technical Guide

Turso operates exclusively in Write-Ahead Log (WAL) mode, where checkpointing serves as the critical mechanism that copies committed transaction frames from the *.db-wal file back into the main database file, ensuring durability while recycling WAL storage space.

The open-source edge database Turso (tursodatabase/turso) implements SQLite's WAL mode as its sole persistence strategy. Unlike traditional SQLite deployments that can toggle between rollback journal and WAL modes, Turso's architecture depends entirely on checkpointing and WAL mode to maintain consistency between the ephemeral write-ahead log and the persistent *.db file.

How Turso Uses Exclusive WAL Mode

In Turso's architecture, the main database file (*.db) never receives writes directly. Instead, every transaction appends frames containing new page data to a separate WAL file (*.db-wal), followed by a commit record that marks the transaction boundary. This design ensures that the primary database file remains unchanged until a checkpoint operation explicitly migrates the data.

This WAL-only durability model means that while transactions are in flight, the main DB stays static. Changes only become durable in the primary file after a successful checkpoint (or manual truncate) operation completes.

The Checkpointing Mechanism

A checkpoint is the process that transfers data from the WAL back into the main database file. This operation serves dual purposes: it persists changes to the primary storage and reclaims space in the WAL file by marking frames as reusable.

According to the Transaction Correctness Guide in the Turso repository, the engine automatically triggers checkpointing when the WAL grows beyond a configurable threshold, defaulting to approximately 1000 pages. The core logic resides in core/storage/wal.rs, where the CheckpointResult struct tracks outcomes including the number of frames back-filled and whether the WAL was truncated.

Checkpointing Process Steps

Because Turso does not use SQLite's traditional -shm index file, the WAL index is maintained in an in-memory hash map (frame_cache) and atomic read-marks (TursoRwLock). The checkpoint process in core/storage/wal.rs follows this sequence:

1. Back-fill pages from the WAL into the DB file.
2. Publish the new back-fill point via publish_backfill.
3. Optionally truncate the WAL file (in TRUNCATE mode).
4. Release locks held for the checkpoint, including the checkpoint lock, read-mark 0, and write lock.

Checkpoint Modes and Concurrency

Turso supports four checkpoint modes inherited from SQLite, each offering different guarantees regarding concurrency and WAL file management:

Mode	Behavior	Blocking Effect
PASSIVE	Copies as many frames as possible without waiting for readers or writers.	No blocking; stops early if readers still need pages.
FULL	Waits for all readers to finish, then copies every frame and syncs the DB file.	Blocks new writers until completion; readers continue.
RESTART	Performs a FULL copy plus restarts the WAL header so the next writer starts at frame 0.	Same as FULL but resets the WAL header.
TRUNCATE	Performs a FULL copy, restarts the WAL, and truncates the WAL file to zero length.	Same as RESTART; empties the WAL file after DB sync.

The PASSIVE mode offers the least interruption but may not complete if the database is busy, while TRUNCATE provides the cleanest slate by zeroing out the WAL file entirely.

Implementing Checkpoints in Code

Developers can invoke checkpoints through SQL PRAGMA statements or Turso's Rust API.

Using SQL PRAGMA

Execute checkpoints directly via SQL commands:

-- Run a passive checkpoint (default)
PRAGMA wal_checkpoint;

-- Full checkpoint, blocks writers until complete
PRAGMA wal_checkpoint(FULL);

-- Restart the WAL (clears the header)
PRAGMA wal_checkpoint(RESTART);

-- Truncate the WAL after copying everything
PRAGMA wal_checkpoint(TRUNCATE);

Using the Rust API

For programmatic control, use the Wal trait and CheckpointMode enum defined in core/storage/wal.rs:

use turso::core::storage::wal::{CheckpointMode, Wal};
use turso::core::storage::pager::Pager;

// Acquire the pager for the DB file
let pager = conn.pager()?;

// Perform a FULL checkpoint
let result = conn.checkpoint(&pager, CheckpointMode::Full)?;
println!("Backfilled {} pages", result.wal_checkpoint_backfilled);

Checkpointing as a Prerequisite for VACUUM

Checkpointing is mandatory before certain maintenance operations. The VACUUM command, implemented in core/vdbe/execute.rs, requires an empty WAL to ensure a clean database file state. The code explicitly invokes conn.checkpoint(CheckpointMode::Truncate) before vacuuming, as demonstrated in tests/integration/query_processing/test_vacuum.rs.

To manually prepare for a VACUUM operation:

use turso::core::storage::wal::CheckpointMode;

// Ensure all changes are checkpointed and WAL is truncated
conn.checkpoint(&pager, CheckpointMode::Truncate)?;
// The WAL file is now empty; VACUUM can proceed safely.

Summary

• Turso runs exclusively in WAL mode, writing all transactions to *.db-wal before the main database file.
• Checkpointing copies frames from the WAL to the *.db file, making changes durable and freeing WAL space.
• Four checkpoint modes (PASSIVE, FULL, RESTART, TRUNCATE) offer trade-offs between concurrency and completeness.
• The implementation lives in core/storage/wal.rs and uses in-memory structures (frame_cache, TursoRwLock) instead of SQLite's -shm file.
• VACUUM and similar operations require a TRUNCATE checkpoint to ensure the WAL is empty before proceeding.

Frequently Asked Questions

What happens if checkpointing fails in Turso?

If a checkpoint fails or is interrupted (particularly in PASSIVE mode), the WAL file retains the uncheckpointed frames. Subsequent transactions continue appending to the WAL, and the next checkpoint attempt will resume copying from where the previous operation left off. The database remains consistent because readers continue using the WAL frames until they are successfully back-filled into the main DB file.

How does Turso's checkpointing differ from standard SQLite?

While Turso implements SQLite's WAL protocol and checkpoint modes, it eliminates the -shm shared-memory index file in favor of an in-memory frame_cache and atomic read-marks (TursoRwLock). This makes Turso's checkpointing logic entirely self-contained within core/storage/wal.rs, optimized for edge deployment scenarios where shared memory might be unreliable or unavailable.

When should I use TRUNCATE versus RESTART checkpoint mode?

Use TRUNCATE when you need to minimize disk space usage or before running maintenance operations like VACUUM, as it zeros out the WAL file after copying. Use RESTART when you want to reset the WAL header for the next writer without the overhead of truncating the file, which is useful in high-throughput scenarios where you want to preserve the file handle but start fresh frame numbering.

Does checkpointing block reads or writes in Turso?

The blocking behavior depends on the CheckpointMode. PASSIVE mode never blocks readers or writers but may incomplete. FULL and TRUNCATE modes block new writers while allowing existing readers to finish, ensuring a consistent snapshot during the copy operation. Readers never block during a checkpoint, though they may read from the WAL file if the required pages haven't been back-filled yet.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}