How celld Manages Concurrent Access to Cells: Epoch-Based Fencing and Request Queuing

celld serializes concurrent access through a deterministic state machine that uses epoch-based fencing to ensure only one writer per cell generation succeeds, while queuing excess requests in permit-based waitlists.

The denoland/celld repository implements a Durable Object runtime that must handle multiple simultaneous requests targeting the same cell without data races. Instead of traditional locking primitives, celld employs a deterministic state machine centered around the on_event entry point in crates/logic/lib.rs, where every request passes through strict authorization checks and permit acquisition before mutating state.

The State Machine Foundation

All state transitions in celld flow through a single function: on_event in crates/logic/lib.rs. This design creates exactly one thread of truth for each cell, preventing race conditions by ensuring that state mutations are applied atomically as Effects. When external events arrive via HTTP or WebSocket handlers, they are routed through crates/celld/runtime.rs and converted into effects that the state machine processes sequentially.

Epoch-Based Fencing for Write Isolation

Each cell carries a generation counter called an epoch that acts as a fence for concurrent writes. The PresenceCell struct defines this mechanism in crates/logic/lib.rs at lines 31-40:

pub struct PresenceCell {
    pub epoch: u64,
    // ... other fields
}

Only one writer is permitted per epoch. When a request attempts to modify a cell, State::request_authorized checks the current epoch and the fenced flag. If the cell is already fenced for another writer, the request is denied with a retry-later response. Once the active writer completes, the epoch increments and the fence lifts, allowing queued requests to proceed.

Request Lifecycle Tracking

The State struct maintains three critical maps that track where each request sits in the authorization pipeline, defined at lines 952-969 in crates/logic/lib.rs:

• request_cells: Maps request IDs to their target cell IDs
• active_requests: Tracks requests currently executing against cells
• request_waiters: Holds requests waiting for authorization or permits

This mapping ensures that celld can precisely control which requests are allowed to transition from "received" to "executing" without overlapping mutations on the same cell.

Permit-Based Activation and Capacity Control

Before executing, a request must acquire an activation permit from the activation_permits pool. If the node has reached max_activations, new requests join the FIFO activation_waiters queue (lines 973-986).

Even after securing an activation permit, requests may face a second queue: capacity_waiters (lines 983-990). This mechanism enforces the max_resident limit, ensuring the node never hosts more active cells than configured. Requests wait here until existing cells idle out or get evicted.

Handling Fenced Writes

When a write operation occurs, celld checks the cell's fenced flag. As implemented around lines 1100-1125 in crates/logic/lib.rs, if a cell is fenced, the write is rejected and the request must retry. This guarantees that only the request holding the current epoch's fence can mutate the cell's durable state. After a successful write, the runtime clears the fence, increments the epoch, and wakes waiters.

Storage-Level Serialization

While the state machine handles logical concurrency, physical durability relies on SQLite. In crates/celld/storage.rs (lines 3-22), each cell maintains a single Connection object. All reads and writes flow through this connection, leveraging SQLite's transaction isolation to further serialize concurrent updates at the storage layer.

Example: Processing Concurrent Requests

When two clients simultaneously fetch the same Durable Object cell, the runtime handles them as follows:

// Concurrent requests to the same cell
async function concurrentFetch(url, cellId) {
  const obj = new DurableObjectNamespace("MyDO").idFromName(cellId);
  const fetch1 = fetch(`${url}/${obj}`, {method: "POST", body: "msg1"});
  const fetch2 = fetch(`${url}/${obj}`, {method: "POST", body: "msg2"});
  return Promise.all([fetch1, fetch2]);
}

Internally, crates/celld/runtime.rs processes these through handle_request:

async fn handle_request(&mut self, req: Request) -> Result<Response> {
    // 1. Authorise – checks fence & epoch
    if !self.state.request_authorized(req.id, req.cell_id) {
        return Err(Error::RetryLater);
    }
    // 2. Acquire activation permit
    self.state.acquire_activation_permit(req.cell_id)?;
    // 3. Process the request (mutates state, writes to storage)
    self.process(req).await?;
    // 4. Release permit & possibly evict idle cell
    self.state.release_activation_permit(req.cell_id);
    Ok(response)
}

The first request acquires the activation permit, passes the epoch check, and sets the fenced flag. The second request fails the authorization check or gets queued in activation_waiters until the first request completes and releases its permit.

Summary

• Epoch-based fencing ensures only one writer per cell generation can modify state at any time, with the epoch field tracking cell generations in PresenceCell.
• Request authorization via request_authorized prevents overlapping writes by checking the fenced flag before allowing execution.
• Permit queues (activation_waiters and capacity_waiters) enforce node-wide limits on concurrent activations and resident cells.
• Atomic state transitions through on_event guarantee that effects apply sequentially, eliminating race conditions without traditional locks.
• SQLite per-cell connections in storage.rs provide durable storage with transaction isolation as a final safety layer.

Frequently Asked Questions

What happens when two requests target the same cell simultaneously?

The first request to pass request_authorized acquires the epoch fence and activation permit, allowing immediate execution. The second request receives a retry-later error or joins the activation_waiters queue, proceeding only after the first request releases its permit and the epoch increments.

How does celld prevent race conditions without using traditional locks?

celld uses a deterministic state machine where all mutations flow through the single on_event function. By tracking request state in maps like active_requests and request_waiters, and by using epoch-based fencing to serialize writers, the system guarantees exactly one thread of truth per cell without mutexes or spinlocks.

What is the purpose of the epoch field in the PresenceCell struct?

The epoch field acts as a generation counter that fences concurrent writes. Only one request can hold the write fence for a specific epoch; once that request completes, the epoch increments, signaling to queued requests that a new writer may proceed. This mechanism lives in crates/logic/lib.rs at lines 31-40.

How does celld handle situations where the node reaches maximum capacity?

When active cells hit max_resident or activation permits exhaust max_activations, celld queues requests in capacity_waiters or activation_waiters respectively. These FIFO queues ensure that requests proceed in order as existing cells idle out (subject to idle_evict_ms) or complete execution, preventing node overload while maintaining fair access.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}