How celld Manages Deployments and Gradual Rollouts Across Its Fleet

celld stores deployment metadata as immutable JSON objects in an S3-compatible bucket, where each release includes a Rollout percentage that nodes use to deterministically decide whether to adopt the new version based on a hash of their unique node ID.

The denoland/celld project implements a coordination-free approach to fleet management that eliminates external consensus services. By treating object storage as the single source of truth, celld manage deployments and gradual rollouts through local evaluation of shared state, allowing each node to independently determine whether to run new or old versions.

The Bucket-Centric Architecture

celld centralizes all deployment state in a shared, S3-compatible bucket rather than relying on a separate control plane or consensus algorithm. The system maintains three critical pieces of state:

• deploy/current.json: Contains the latest committed Worker/Durable Object bundle version
• Rollout metadata: A percentage field indicating what portion of the fleet should run the new version
• Cell ownership: SQLite databases (cells) are owned by single nodes through an object-storage compare-and-swap protocol

When a node starts, it reads the current deployment from deploy/current.json in the bucket. This file represents the latest successfully committed deployment that serves as the authority for the entire fleet.

Deterministic Rollout Logic

Each deployment carries a Rollout struct defined in crates/celld/protocol.rs:

pub struct Rollout {
    pub percent: u8,
}

During the control-plane reconciliation loop in crates/celld/control_plane.rs, nodes evaluate whether to adopt the new deployment. The logic checks deployment.pointer.rollout.percent against a deterministic hash of the node's own identifier:

// Simplified control-plane logic
if deployment.pointer.rollout.percent != 100 {
    let h = hash(node_id) % 100;
    if h < deployment.pointer.rollout.percent {
        load_new_bundle(&deployment);
    }
}

If the computed hash modulo 100 falls below the configured rollout percentage, the node loads the new bundle. Otherwise, it continues running the previous version. This approach ensures that exactly the specified percentage of the fleet adopts the new version without requiring central coordination.

Step-by-Step Deployment Workflow

Operators manage rollouts through the celld deploy command, progressively increasing the rollout percentage to shift traffic:

1. Initial deployment at 0%: Deploy the bundle without activating it

   celld deploy . \
     --bucket s3://my-cells-bucket \
     --rollout 0

2. Canary rollout: Enable the version for a subset of nodes

   celld deploy . \
     --bucket s3://my-cells-bucket \
     --rollout 25

3. Full rollout: Complete the deployment across the entire fleet

   celld deploy . \
     --bucket s3://my-cells-bucket \
     --rollout 100

All nodes watch the bucket for changes to current.json. When updated, each node independently recalculates its eligibility and transitions to the new version if the hash check passes.

Key Source Files

The implementation spans several core files:

• crates/celld/protocol.rs: Defines the Rollout struct with the percent field used in deployment metadata
• crates/celld/deploy.rs: Implements the celld deploy command, writes deploy/current.json including the rollout configuration
• crates/celld/control_plane.rs: Contains the reconciliation loop that reads deployments and evaluates rollout.percent to determine version adoption
• README.md: Documents the high-level workflow of bucket-based coordination and gradual rollouts

Summary

• celld uses an S3-compatible bucket as the single source of truth for deployment state, eliminating external consensus requirements
• The Rollout struct contains a percent field (0-100) that controls what portion of the fleet runs the new version
• Nodes use deterministic hashing of their node ID to decide independently whether to adopt new deployments
• The celld deploy command with --rollout flag enables progressive rollouts from 0% to 100%
• Object-storage compare-and-swap protocols handle cell ownership transfers during version transitions

Frequently Asked Questions

How does celld ensure exactly the specified percentage of nodes adopt a new version?

Each node computes a deterministic hash of its unique identifier modulo 100. If this value is less than the rollout.percent field in the deployment metadata, the node adopts the new version. Because the hash is deterministic and the percentage is shared via the central bucket, the law of large numbers ensures the fleet converges to the specified distribution without central coordination.

What happens to cells when a node running an old version shuts down?

Cells (SQLite databases) are owned by single nodes through the bucket's compare-and-swap protocol. When a node shuts down, the ownership mechanism reassigns its cells to nodes currently running the new version, ensuring seamless continuity during gradual rollouts.

Why doesn't celld use a traditional control plane or consensus service?

The design treats the S3-compatible bucket as the coordination point. By storing immutable deployment JSON objects and using deterministic local evaluation, celld avoids the complexity and operational overhead of external consensus algorithms like Raft or Paxos. Each node makes independent decisions based on shared state, simplifying the architecture while maintaining consistency.

Can you adjust the rollout percentage without changing the code bundle?

Yes. Operators simply run celld deploy with the desired --rollout value against the same bundle. This updates deploy/current.json with a new Rollout percentage, causing nodes to recalculate their eligibility without requiring a new code build.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}