How Cell Ownership Records Transition Between Owned, Released and Sticky States in celld

Cell ownership records in celld transition between owned, released, and sticky states through atomic compare-and-swap (CAS) operations on S3-compatible storage, governed by the OwnershipOnEvict policy that determines whether evicted cells become unowned or remain pinned to their current node.

The denoland/celld runtime manages distributed Durable Object instances—called "cells"—by tracking cell ownership records in a shared object-storage bucket. These records transition through three distinct states to coordinate exclusive access across a fleet of nodes, with each transition governed by specific eviction policies and atomic storage operations. Understanding these state transitions is essential for configuring celld deployments that balance high availability against cache locality and fast wake-up times.

Understanding Cell Ownership States

Each cell maintains an ownership record in the shared S3-compatible bucket consisting of a node ID and epoch number (see OwnerRecord in crates/logic/lib.rs). These records exist in one of three states:

• Owned: The record contains a specific node ID, granting exclusive serving rights to that node.
• Released: The record contains no node ID (None), indicating the cell is unowned and available for acquisition by any node.
• Sticky: The record retains the original node ID after eviction, allowing the same node to reacquire ownership immediately upon wake-up without a full hand-off.

The OwnershipOnEvict Policy

The eviction policy determines whether a cell becomes released or sticky when evicted from memory. This behavior is expressed by the OwnershipOnEvict enum defined in crates/logic/lib.rs at lines 45-53:

/// What an evicted cell's ownership record should say.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub enum OwnershipOnEvict {
    /// Publish the cell as unowned so any node may take it next.
    #[default]
    Release,
    /// Keep the record, so a same-node wake can reuse the local snapshot.
    Sticky,
}

When OwnershipOnEvict::Release is active, evicted cells become available to the entire fleet. When set to Sticky, celld preserves the node affinity to minimize cold-start latency for that specific cell.

State Transition Mechanics

All state transitions rely on the bucket's compare-and-swap (CAS) semantics to guarantee atomic updates without requiring a separate consensus protocol.

Owned to Released

When the eviction policy is Release, the core logic calls ownership.release_owner(&cell, epoch) in crates/celld/main.rs (around line 1500). This operation issues a CAS request to the bucket that sets the node field of the OwnerRecord to None. Once applied, the cell becomes unowned and eligible for acquisition by any node in the fleet.

Owned to Sticky

If the policy is Sticky, the same release_owner call still performs a CAS, but the new OwnerRecord retains the current node ID with the epoch unchanged. The cell remains effectively "reserved" for the original node, enabling local wake-up without the network overhead of a full ownership hand-off.

Released to Owned

Any node attempting to serve a request for an unowned cell performs an ownership acquire via ownership.cas_owner(&cell, guard, epoch). This operation attempts to atomically write the acquiring node's ID into the record. On success (CasOutcome::Applied), the state transitions from Released to Owned, granting exclusive access.

Sticky to Owned

When a record is already sticky (the node field matches the local node ID), the node already possesses valid ownership. Subsequent request handling proceeds without additional acquisition steps, as the local snapshot remains authoritative.

Implementation Details

The coordination logic spans multiple crates and files within the celld repository:

• crates/logic/lib.rs (lines 45-53): Defines the OwnershipOnEvict enum and the decision logic at line 3421 that checks self.config.ownership_on_evict == OwnershipOnEvict::Release.
• crates/celld/main.rs (around line 1500): Invokes ownership.release_owner(&cell, epoch) during eviction handling.
• crates/celld/main.rs (line 1418): Reads current ownership records via ownership.read_owner(&cell).
• crates/celld/ownership_store.rs: Implements the S3-backed storage adapter with cas_owner and release_owner methods that execute the actual CAS operations.

Practical Code Examples

Configure the eviction policy to keep cells sticky on eviction:

let mut config = Config::default();
config.ownership_on_evict = OwnershipOnEvict::Sticky; // keep sticky on eviction

Force a release of a specific cell during manual shedding:

let epoch = current_epoch();
let result = ownership.release_owner(&cell_id, epoch).await?;
if let Ok(CasOutcome::Applied) = result {
    println!("Cell {} ownership released (now unowned)", cell_id);
}

Acquire ownership of an unowned cell:

let guard = ...; // generated guard (etag) from a prior read
let result = ownership.cas_owner(&cell_id, guard, new_epoch).await?;
match result {
    Ok(CasOutcome::Applied) => println!("Acquired ownership of {}", cell_id),
    Ok(CasOutcome::Rejected) => println!("Ownership race lost for {}", cell_id),
    Err(e) => eprintln!("CAS error: {}", e),
}

Summary

• Cell ownership records track node assignment and epoch information in S3-compatible storage.
• The OwnershipOnEvict enum determines whether evicted cells become released (Release) or sticky (Sticky).
• Owned → Released transitions clear the node ID via CAS, making cells available for fleet-wide acquisition.
• Owned → Sticky transitions preserve the node ID, enabling fast local wake-ups without hand-off overhead.
• Released → Owned transitions occur when nodes successfully execute cas_owner, writing their node ID into the empty record.
• All transitions rely on atomic CAS operations to prevent split-brain scenarios without requiring external consensus.

Frequently Asked Questions

What is the default eviction policy in celld?

The default eviction policy is Release, as indicated by the #[default] attribute on the OwnershipOnEvict enum in crates/logic/lib.rs. This means evicted cells become unowned by default, allowing any node in the fleet to acquire them upon the next request.

How does celld prevent split-brain scenarios during ownership transitions?

Celld relies on the compare-and-swap (CAS) semantics provided by the underlying S3-compatible object storage. When transitioning states, the system includes a guard (etag) from a prior read that must match the current storage version. If the record changed between read and write, the CAS is rejected, ensuring at most one node can successfully claim ownership at any moment.

When should I use Sticky vs Release eviction policies?

Use Sticky when optimizing for latency on frequently accessed cells that typically return to the same node, as it eliminates network overhead and serialization costs during wake-up. Use Release when running under memory pressure or when cells are rarely reused by the same node, as it allows the fleet to distribute load more evenly and prevents memory fragmentation on individual nodes.

What happens to the epoch during ownership transitions?

The epoch remains unchanged during Owned → Sticky transitions to preserve the logical timeline for that cell's state. During Owned → Released transitions, the epoch is preserved in the record structure but the node ID becomes None. When a new node acquires a released cell, it typically increments or validates the epoch as part of the cas_owner operation to ensure consistency with the cell's state machine.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}