# How Multi-Process WAL Coordination Works in Turso: The .tshm Sidecar Explained

> Discover how Turso engineers multi-process WAL coordination with its .tshm sidecar. Learn about byte-range locking and shared snapshots for seamless synchronization.

- Repository: [Turso Database/turso](https://github.com/tursodatabase/turso)
- Tags: internals
- Published: 2026-06-23

---

**Turso coordinates concurrent database access across multiple OS processes through a memory-mapped `.tshm` sidecar file that implements byte-range locking and a shared snapshot header to synchronize WAL state.**

Turso's experimental multi-process WAL feature allows several independent processes to safely open and modify the same SQLite database file simultaneously. According to the tursodatabase/turso source code, this coordination relies on a sidecar file named `<db>.db-tshm` (Turso Shared Heap Memory) that stores lock state, snapshot metadata, and a shared frame index in memory-mapped shared memory.

## The .tshm File Architecture

The sidecar file stores three distinct categories of state in a structured memory layout defined in [`core/storage/shared_wal_coordination.rs`](https://github.com/tursodatabase/turso/blob/main/core/storage/shared_wal_coordination.rs):

- **WAL Snapshot Header** (`SharedWalCoordinationHeader`): Tracks `max_frame`, checksums, salts, and checkpoint counters in a fixed header region at the beginning of the mapping.
- **Ownership Slots**: Enforces exclusive access through byte-range locks. Byte offset **0** indicates the lifetime lock (exclusive vs. shared mode), offset **1** is the writer slot, offset **2** is the checkpointer slot, and offsets **3 and above** represent reader slots (one per concurrent read transaction).
- **Shared Frame Index**: An append-only `FrameIndexBlockMapping` that follows the fixed header, enabling any process to resolve a page number to its newest WAL frame without scanning the entire log.

## Opening and Initializing the Shared Memory

When a process opens a database with `--experimental-multiprocess-wal`, Turso invokes `MappedSharedWalCoordination::create_or_open` in [`core/storage/shared_wal_coordination.rs`](https://github.com/tursodatabase/turso/blob/main/core/storage/shared_wal_coordination.rs). This function:

1. **Detects the open mode** by attempting an exclusive lock on byte 0. Success indicates the process is the **Exclusive** opener; failure means another process already holds the lifetime lock, forcing **MultiProcess** mode.
2. **Creates or resizes** the `.tshm` file to accommodate the fixed header plus an initial 64 KB frame-index block.
3. **Memory-maps** the header region and lazily maps additional index blocks on demand.
4. **Registers** the mapping in `PROCESS_LOCAL_COORDINATION_OPENS` to ensure duplicate opens within the same process share a single `Arc<MappedSharedWalCoordination>` instance.

## Cross-Process Locking Mechanisms

Turso enforces writer, checkpointer, and reader exclusivity using byte-range locks on the `.tshm` file descriptor. The implementation adapts to platform capabilities:

- **Linux**: Uses `F_OFD_SETLK` (open file description locks), which survive `dup()` calls and work correctly across different `open()` invocations by distinct processes.
- **macOS and other Unix systems**: Falls back to classic `F_SETLK` (process-level locks), requiring additional probing to detect stale owners.

When acquiring a lock, processes write a `SharedOwnerRecord` containing their PID and instance ID into shared fields (such as `writer_owner`) for diagnostic visibility. If lock acquisition fails, the caller receives `SQLITE_BUSY` until the owner releases the byte or is reclaimed as stale.

## Publishing Consistent Snapshots

Writers disseminate committed transaction state through a **sequence lock** (`snapshot_seq`) in the shared header:

1. The writer increments `snapshot_seq` to an **odd** value before modifying header fields (`max_frame`, `nbackfills`, etc.).
2. After updating the data, the writer increments it back to an **even** value.
3. Readers calling `snapshot()` spin-wait until the sequence is even, then read the header, verifying the sequence matches before and after the read to ensure consistency.

## The Shared Frame Index

Every WAL frame commit appends a `SharedWalFrameIndexEntry` to the shared index, which is divided into fixed-size blocks holding **4096 entries** each. The writer:

- Locks `frame_index_publish_lock` (a process-local mutex) to serialize appends.
- Updates the block hash table and publishes the new `frame_index_len`.
- Readers locate pages by scanning block hash tables rather than traversing the WAL.

If the index overflows (all blocks filled), Turso falls back to a full WAL scan and rebuilds the index from disk.

## Cleanup and Failure Handling

When `MappedSharedWalCoordination::drop` executes (process exit or database close):

- Releases all held byte-range locks (writer, checkpointer, reader slots).
- Clears ownership bits in shared memory.
- Decrements the per-process open count, removing the entry from the registry if zero.

If the process holds `is_last_process_mapping` (the final lifetime lock holder), Turso may perform a final checkpoint to prevent dangling frames.

**Mode conflicts** are explicitly prevented: opening a database with the experimental flag while another process holds a legacy exclusive lock (or vice versa) returns a clear error. Read-only opens gracefully fall back to legacy read-only WAL access if no `.tshm` file exists.

## Practical Examples

### Enabling Multi-Process WAL via CLI

All processes must use the same flag, or they will receive a conflict error:

```bash
turso --experimental-multiprocess-wal mydb.db

```

### Configuring from Rust

```rust
use turso::Builder;

let db = Builder::new_local("mydb.db")
    .experimental_multiprocess_wal(true)
    .build()?;

```

### Low-Level Lock Acquisition (Internal API)

```rust
use std::sync::Arc;
use turso::io::{IO, SharedWalLockKind};

fn acquire_writer(io: &Arc<dyn IO>, tshm_path: &std::path::Path) -> turso::Result<()> {
    let file = io.open_shared_wal_file(tshm_path.to_str().unwrap())?;
    file.shared_wal_lock_byte(1, false, SharedWalLockKind::LinuxOfd)?;
    Ok(())
}

```

### Reading Consistent Snapshots

```rust
use turso::core::storage::shared_wal_coordination::MappedSharedWalCoordination;

fn current_snapshot(coord: &MappedSharedWalCoordination) -> SharedWalCoordinationHeader {
    coord.snapshot()
}

```

### Registering a Reader Slot

```rust
fn register_reader(coord: &MappedSharedWalCoordination, max_frame: u64) -> Result<usize> {
    let slot = coord.acquire_reader_slot(max_frame)?;
    Ok(slot)
}

```

## Summary

- The `.tshm` sidecar file acts as a memory-mapped coordination plane for multiple processes accessing the same database.
- **Byte-range locks** (Linux OFD or POSIX) on specific byte offsets (0=lifetime, 1=writer, 2=checkpointer, 3+=readers) enforce exclusive access to critical roles.
- A **sequence lock** (`snapshot_seq`) in the shared header ensures readers see consistent snapshots even as writers update `max_frame` and other fields atomically.
- The **append-only frame index** (4096 entries per block) eliminates full WAL scans for page resolution.
- Cleanup logic in `MappedSharedWalCoordination::drop` reclaimes stale locks and triggers final checkpoints when the last process exits.

## Frequently Asked Questions

### What is the .tshm file in Turso?

The `.tshm` (Turso Shared Heap Memory) file is a sidecar database file created alongside your main database when multi-process WAL mode is enabled. It stores shared memory structures—including the snapshot header, ownership slots, and frame index—that multiple processes map into their address space to coordinate concurrent access.

### How does Turso handle writer conflicts between processes?

Turso assigns writer ownership using byte-range locks on offset **1** of the `.tshm` file. Only one process can hold this lock at a time. If another process attempts to write while the lock is held, it receives `SQLITE_BUSY` until the lock becomes available. Each successful lock acquisition also records a `SharedOwnerRecord` with the process PID for diagnostics.

### What happens if a process crashes while holding a lock?

On Linux, open file description (OFD) locks automatically release when the file descriptor closes (even on crash). On macOS and other Unix systems using classic `F_SETLK` locks, subsequent acquire attempts probe the lock byte. If the probe succeeds, the previous owner is deemed dead and Turso clears its slot, allowing the new process to proceed.

### Can I use multi-process WAL with read-only connections?

Yes. If a read-only connection requests experimental multi-process mode but no `.tshm` file exists (because no writer has initialized it), Turso automatically falls back to the legacy read-only WAL path. This preserves compatibility for diagnostic tools that only need to inspect the database without participating in the coordination protocol.