How celld Achieves Horizontal Scaling Without a Traditional Control Plane or Consensus

Celld achieves horizontal scaling by deploying a fleet of identical, stateless nodes that synchronize via an append-only LTX log stored in shared object storage, eliminating the need for leader election, consensus protocols, or persistent control planes.

The denoland/celld project implements a distributed serverless runtime designed specifically to achieve horizontal scaling without a traditional control plane or consensus mechanism. Rather than relying on Raft, Paxos, or centralized coordinators, celld operates as a symmetric mesh of peers that independently reconcile state from immutable log segments stored in cloud object buckets.

Stateless Architecture and Peer Equality

Every celld node is identical and stateless, capable of joining or leaving the fleet without coordination. No single node holds authoritative state, and the system requires no leader election to function. According to the source code in crates/celld/peer_probe.rs, nodes maintain a bi-directional probe mechanism that continuously exchanges health pings and updates a shared PeerMap, ensuring mesh connectivity without hierarchy.

Three Mechanisms for Control-Plane-Free Scaling

Celld replaces heavyweight coordination with three lightweight components that work together to enable elastic scaling.

Optional Managed Control Plane (Bootstrap Only)

The managed control plane, implemented in crates/celld/control_plane.rs, provides only initial configuration and credential rotation. It does not maintain cluster state or coordinate replication. The ManagedRuntimeState enum tracks whether the node is connected, disconnected, or if the control plane is unavailable, while the report_managed_runtime_state function logs these transitions.

If the control plane becomes unreachable after startup, nodes fall back to bucket-only serving via ManagedRuntimeState::ControlPlaneUnavailable and continue operating independently. Credential revocation triggers a graceful restart via restart_process (lines 51-55 of control_plane.rs) to fetch fresh credentials, but this requires no distributed coordination.

Peer Discovery and Bi-Directional Probing

Nodes locate peers via DNS or static lists without central registration. The probe loop in crates/celld/peer_probe.rs continuously verifies connectivity and exchanges node identifiers, maintaining a full-mesh topology where any node can forward requests to any other. This symmetric design eliminates the need for service discovery services or load balancer configuration.

LTX Log and Shared Bucket Storage

All mutations are recorded in an append-only LTX log implemented in crates/celld/ltx_repl.rs. The crates/celld/bucket.rs module provides Bucket::get and Bucket::put abstractions over S3, Azure Blob, or Cloudflare R2. Each node independently pulls log segments from this shared bucket and applies them locally.

Because the log is immutable, nodes can replay segments in any order, achieving eventual consistency without consensus rounds. The crates/celld/replication.rs module handles streaming these segments and reconciling conflicts via CRDT-style merge rules.

How the Replication Flow Works

The system achieves convergence through six distinct phases defined in the source code:

1. Bootstrapping – On startup (startup.rs), nodes optionally contact the control plane to fetch a short-lived token and fleet bucket URL, then immediately operate independently.

2. Mesh Formation – Using peer_probe.rs, nodes advertise their presence and build a symmetric peer mesh where all nodes are equals.

3. Mutation Logging – Application writes go to local SQLite and are simultaneously appended to the LTX log.

4. Bucket Upload – The replication.rs component uploads new log segments to the shared bucket using Bucket::put.

5. Background Synchronization – Each node periodically calls Bucket::get to download new segments, re-applies them locally, and updates state deterministically.

6. Graceful Degradation – If credentials expire, the node triggers restart_process to refresh tokens; if the control plane is down, nodes continue serving from the bucket indefinitely.

Practical Implementation Examples

Connecting from JavaScript

The WebSocket client connects to any node in the fleet—the underlying replication guarantees all nodes converge to the same state:

// examples/wsclient/index.js
import { connect } from "https://deno.land/x/celld/ws_client.ts";

async function run() {
  // Connect to any node in the fleet
  const client = await connect("wss://my-fleet.example.com");

  // Write to shared storage
  await client.exec(`
    INSERT INTO kv (key, value) VALUES ('greeting', 'Hello, world!')
    ON CONFLICT(key) DO UPDATE SET value = excluded.value;
  `);

  // Read from any peer (request may be served by different node)
  const rows = await client.query("SELECT value FROM kv WHERE key = 'greeting'");
  console.log("Fetched:", rows[0].value);
}

run();

Starting a Stateless Node

Launch a new node by pointing it to the shared bucket. No cluster registration or join protocol is required:

# Environment-based configuration

CELLD_NODE=my-node-id \
CELLD_BUCKET=s3://my-fleet-bucket \
CELLD_CONTROL_PLANE_URL=https://celld.dev \
celld start

• CELLD_BUCKET specifies the shared LTX log location (required)
• CELLD_CONTROL_PLANE_URL is optional; omitting it runs the node completely autonomously

The node immediately begins probing the bucket for log segments and joining the peer mesh via peer_probe.rs.

Summary

• Celld achieves horizontal scaling without a traditional control plane or consensus by using stateless nodes that pull state from shared storage.
• The optional control plane (control_plane.rs) handles only bootstrap credentials via ManagedRuntimeState, not runtime coordination.
• Peer discovery operates via bi-directional probing (peer_probe.rs) in a symmetric mesh with no leader.
• LTX log replication (ltx_repl.rs, replication.rs) stores mutations in immutable, append-only segments within a shared bucket (bucket.rs), allowing nodes to converge without consensus algorithms.
• New nodes join by simply reading the complete log history; departing nodes require no graceful shutdown coordination.

Frequently Asked Questions

Does celld require a control plane to operate?

No. The managed control plane is optional and only used during startup for credential provisioning and environment configuration. As implemented in crates/celld/control_plane.rs, nodes can run entirely autonomously by specifying only CELLD_BUCKET. If the control plane becomes unavailable after startup, nodes transition to ManagedRuntimeState::ControlPlaneUnavailable and continue serving traffic using bucket storage alone.

How does celld ensure consistency without a consensus algorithm?

Celld uses an append-only LTX log stored in shared object storage. Because log entries are immutable, every node can pull and apply them independently in any order. The replication.rs module reconciles conflicts using CRDT-style merge rules, guaranteeing that all replicas eventually converge to the same state without requiring Paxos, Raft, or two-phase commit protocols.

What happens when a new node joins the fleet?

Adding capacity requires no coordination. A new node simply launches with the same CELLD_BUCKET URL, pulls the complete log history from the bucket via Bucket::get, joins the peer mesh through peer_probe.rs, and begins serving traffic immediately. There is no "join consensus" or cluster membership change protocol.

How does the system handle node failures or network partitions?

Node failures require no failover logic. When a node departs, the remaining peers stop receiving its probes via the probe loop in peer_probe.rs and remove it from their PeerMap. During network partitions, nodes continue writing to their local LTX logs and upload segments to the shared bucket; when connectivity returns, all nodes synchronize missed segments independently, relying on the immutability of the log to resolve conflicts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}