Turso Multi-Process WAL Coordination with .tshm Sidecar: Implementation Guide

Turso uses a memory-mapped .tshm sidecar file to coordinate write-ahead-log access across multiple OS processes, enabling safe concurrent reads and writes without per-process WAL scanning.

The tursodatabase/turso repository implements an experimental multi-process write-ahead-log (WAL) coordination mechanism that allows several operating system processes to share a single database file safely. At the heart of this system lies the .tshm sidecar, a shared-memory file that stores a compact data structure tracking WAL state, writer ownership, and a shared page-to-frame index. This architecture eliminates the need for each process to scan the entire WAL independently, significantly improving performance in multi-process workloads.

What Is the .tshm Sidecar?

The .tshm sidecar is a shared-memory coordination file created alongside the main database when multi-process WAL mode is enabled. When a database is opened with the --experimental-multiprocess-wal flag, Turso creates or reuses a file named <dbname>.db-tshm in the same directory as the database.

According to the source code in core/storage/shared_wal_coordination.rs, this file is memory-mapped by every participating process and contains:

• A fixed header with magic constants, version numbers, and snapshot metadata (max_frame, checkpoint_seq, page size)
• Lock bytes for coordinating writers, checkpointers, and readers
• Owner records tracking which OS process holds which slot
• The shared page-to-frame index for fast WAL lookups

How the Sidecar Coordinates Multi-Process Access

Shared-Memory Layout and Header Structure

The sidecar file begins with a fixed header that contains a magic constant and version number to validate the file format. The header stores critical snapshot metadata including the current max_frame, nbackfills, checkpoint_seq, and the database page size. This layout is defined in core/storage/shared_wal_coordination.rs where the data structures specify the exact byte offsets for each field.

When Turso opens a database, it probes the .tshm file. If the file is fresh (size 0), the sidecar initializes with default values. If the file exists, Turso validates the persisted header against expected values. Validation failures behave differently depending on the open mode: under an Exclusive open, Turso sanitizes the backfill proof and reinitializes the header, whereas under a MultiProcess open, an invalid header results in an error because another live process may rely on that state.

Lock Byte Layout and Platform-Specific Locking

The sidecar uses a byte-range locking scheme to coordinate access between processes. The lock byte layout assigns specific meaning to each byte:

• Byte 0: The lifetime lock distinguishing Exclusive opens (no other process holds the file) from MultiProcess opens (another process already holds the sidecar)
• Byte 1: The writer lock
• Byte 2: The checkpointer lock
• Bytes 3+: Individual lock bytes for each reader slot

The implementation differs by platform to handle different fcntl semantics. On Linux, Turso uses open-file-description locks (F_OFD_SETLK) via the SharedWalOwnershipMode enum, which are independent per open() call and do not conflict with threads in the same process. On macOS and other Unix systems, it falls back to classic F_SETLK process-scoped locks, which require additional coordination to avoid conflicts between connections in the same process.

Ownership Bookkeeping and Process-Local State

To prevent a process from deadlocking against itself when using process-scoped locks, Turso maintains owner records in the shared slots containing the PID and instance ID. Inside each process, a ProcessLocalOwnershipState mirrors these slots, ensuring that sibling connections within the same process respect each other's ownership without attempting to acquire conflicting locks. This separation is vital because F_SETLK locks are per-process, not per-file-descriptor, making internal tracking necessary to avoid self-deadlock.

Shared Page-to-Frame Index and Snapshot Publishing

The Append-Only Index Structure

To avoid a full WAL scan on every read, the sidecar maintains an append-only index mapping database pages to the most recent WAL frame that modified them. The index grows in fixed-size blocks with a capacity of 4,096 entries per block, using an open-addressing hash table for fast lookups. This index is protected by a per-process frame_index_publish_lock and is lazily memory-mapped as needed to conserve resources.

Lock-Free Snapshot Reading

When a writer commits a new WAL frame, it updates the shared header inside a sequence lock (snapshot_seq). Readers spin on this odd/even value to obtain a coherent snapshot of fields such as max_frame, nbackfills, and checkpoint_seq. The snapshot() method in shared_wal_coordination.rs demonstrates this lock-free read pattern, allowing readers to access current WAL metadata without blocking writers.

// Accessing the current snapshot from any process
let snapshot = coordination.snapshot();
println!("Current WAL max frame: {}", snapshot.max_frame);

Recovery, Re-opening, and Graceful Shutdown

On process startup, Turso probes the .tshm file to determine initialization state. If the file is fresh, it initializes the header; otherwise, it validates the persisted structure. When the last process holding a sidecar mapping exits, Turso detects this condition through the is_last_process_mapping() method, which checks if the lifetime lock (byte 0) can be reacquired. This mirrors SQLite’s "last connection cleans up shared state" behavior, ensuring that temporary files are properly managed when no processes remain.

// Detecting the last process before shutdown
if coordination.is_last_process_mapping() {
    // This is the final process holding the .tshm sidecar
    // Perform necessary cleanup before exit
}

Enabling Multi-Process WAL Mode

Command-Line Interface

To enable the experimental feature when launching the server or CLI REPL, add the flag documented in docs/manual.md at line 158:

tursodb --experimental-multiprocess-wal path/to/database.db

Rust API Integration

The high-level API in core/storage/wal.rs integrates with the shared coordination when multiprocess_wal is enabled in the configuration:

use turso::Database;
use turso::OpenOptions;

// High-level API
let db = Database::open(
    OpenOptions::new()
        .path("example.db")
        .multiprocess_wal(true)   // enables the .tshm sidecar
)?;

For low-level control, the MappedSharedWalCoordination::create_or_open method in core/storage/shared_wal_coordination.rs creates the sidecar directly:

use std::sync::Arc;
use turso::io::IO;
use turso::storage::shared_wal_coordination::MappedSharedWalCoordination;

fn open_shared_coordination(io: &Arc<dyn IO>, db_path: &std::path::Path) -> anyhow::Result<()> {
    // reader_slot_count is typically a multiple of 64; 256 is a common default
    let coordination = MappedSharedWalCoordination::create_or_open(
        io,
        &db_path.with_extension("db-tshm"),
        256,
    )?;
    
    // The returned coordination can be used to register readers, obtain snapshots, etc.
    Ok(())
}

Summary

• The .tshm sidecar is a memory-mapped coordination file that replaces SQLite’s per-process WAL scan in multi-process scenarios, located in core/storage/shared_wal_coordination.rs.
• It provides fast, lock-free snapshots of WAL metadata and a shared page-to-frame index with 4,096-entry blocks to serve readers without scanning the WAL.
• Ownership is maintained via byte-range locks (OFD locks on Linux, F_SETLK on macOS) and process-local registries (ProcessLocalOwnershipState) to avoid conflicts between connections in the same process.
• The feature is controlled by the --experimental-multiprocess-wal flag and is intended for workloads where many short-lived processes need concurrent read/write access to the same Turso database.

Frequently Asked Questions

What is the .tshm file format?

The .tshm file is a binary shared-memory file containing a fixed header with magic constants and snapshot metadata, followed by lock bytes, owner records (PID + instance ID), and an append-only page-to-frame index. The layout is defined in core/storage/shared_wal_coordination.rs and uses open-addressing hash tables with 4,096-entry blocks for the index structure.

How does Turso handle locking on Linux vs macOS?

On Linux, Turso uses open-file-description locks (F_OFD_SETLK) which are scoped to the file descriptor rather than the process, allowing multiple threads in the same process to hold different locks without conflict. On macOS and other Unix systems, it falls back to classic process-scoped locks (F_SETLK), requiring the ProcessLocalOwnershipState to track which connections within a process hold which slots to prevent self-deadlock.

What happens when the last process exits?

When the final process holding a mapping to the .tshm file exits, Turso detects this via the is_last_process_mapping() method, which attempts to reacquire the lifetime lock on byte 0. If successful, the process knows it is the last one and can perform cleanup operations, similar to SQLite’s last-connection-cleanup pattern.

Is multi-process WAL production ready?

No, multi-process WAL coordination is experimental. The feature is guarded by the --experimental-multiprocess-wal flag and documented in docs/manual.md and docs/sql-reference/multiprocess-access.mdx as not being production-ready. It is intended for testing and specific workloads where many short-lived processes require concurrent database access.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}