# How celld Achieves Horizontal Scaling Without a Traditional Control Plane or Consensus

> Discover how celld scales horizontally using stateless nodes and an LTX log, bypassing traditional control planes and consensus protocols for efficient operation.

- Repository: [Deno/celld](https://github.com/denoland/celld)
- Tags: architecture
- Published: 2026-08-10

---

**Celld achieves horizontal scaling by deploying a fleet of identical, stateless nodes that synchronize via an append-only LTX log stored in shared object storage, eliminating the need for leader election, consensus protocols, or persistent control planes.**

The `denoland/celld` project implements a distributed serverless runtime designed specifically to achieve horizontal scaling without a traditional control plane or consensus mechanism. Rather than relying on Raft, Paxos, or centralized coordinators, celld operates as a symmetric mesh of peers that independently reconcile state from immutable log segments stored in cloud object buckets.

## Stateless Architecture and Peer Equality

Every celld node is identical and stateless, capable of joining or leaving the fleet without coordination. No single node holds authoritative state, and the system requires no leader election to function. According to the source code in [`crates/celld/peer_probe.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_probe.rs), nodes maintain a **bi-directional probe** mechanism that continuously exchanges health pings and updates a shared `PeerMap`, ensuring mesh connectivity without hierarchy.

## Three Mechanisms for Control-Plane-Free Scaling

Celld replaces heavyweight coordination with three lightweight components that work together to enable elastic scaling.

### Optional Managed Control Plane (Bootstrap Only)

The managed control plane, implemented in [`crates/celld/control_plane.rs`](https://github.com/denoland/celld/blob/main/crates/celld/control_plane.rs), provides only initial configuration and credential rotation. It does not maintain cluster state or coordinate replication. The `ManagedRuntimeState` enum tracks whether the node is connected, disconnected, or if the control plane is unavailable, while the `report_managed_runtime_state` function logs these transitions.

If the control plane becomes unreachable after startup, nodes fall back to **bucket-only serving** via `ManagedRuntimeState::ControlPlaneUnavailable` and continue operating independently. Credential revocation triggers a graceful restart via `restart_process` (lines 51-55 of [`control_plane.rs`](https://github.com/denoland/celld/blob/main/control_plane.rs)) to fetch fresh credentials, but this requires no distributed coordination.

### Peer Discovery and Bi-Directional Probing

Nodes locate peers via DNS or static lists without central registration. The `probe` loop in [`crates/celld/peer_probe.rs`](https://github.com/denoland/celld/blob/main/crates/celld/peer_probe.rs) continuously verifies connectivity and exchanges node identifiers, maintaining a full-mesh topology where any node can forward requests to any other. This symmetric design eliminates the need for service discovery services or load balancer configuration.

### LTX Log and Shared Bucket Storage

All mutations are recorded in an append-only **LTX log** implemented in [`crates/celld/ltx_repl.rs`](https://github.com/denoland/celld/blob/main/crates/celld/ltx_repl.rs). The [`crates/celld/bucket.rs`](https://github.com/denoland/celld/blob/main/crates/celld/bucket.rs) module provides `Bucket::get` and `Bucket::put` abstractions over S3, Azure Blob, or Cloudflare R2. Each node independently pulls log segments from this shared bucket and applies them locally.

Because the log is immutable, nodes can replay segments in any order, achieving **eventual consistency** without consensus rounds. The [`crates/celld/replication.rs`](https://github.com/denoland/celld/blob/main/crates/celld/replication.rs) module handles streaming these segments and reconciling conflicts via **CRDT-style merge rules**.

## How the Replication Flow Works

The system achieves convergence through six distinct phases defined in the source code:

1. **Bootstrapping** – On startup ([`startup.rs`](https://github.com/denoland/celld/blob/main/startup.rs)), nodes optionally contact the control plane to fetch a short-lived token and fleet bucket URL, then immediately operate independently.

2. **Mesh Formation** – Using [`peer_probe.rs`](https://github.com/denoland/celld/blob/main/peer_probe.rs), nodes advertise their presence and build a symmetric peer mesh where all nodes are equals.

3. **Mutation Logging** – Application writes go to local SQLite and are simultaneously appended to the LTX log.

4. **Bucket Upload** – The [`replication.rs`](https://github.com/denoland/celld/blob/main/replication.rs) component uploads new log segments to the shared bucket using `Bucket::put`.

5. **Background Synchronization** – Each node periodically calls `Bucket::get` to download new segments, re-applies them locally, and updates state deterministically.

6. **Graceful Degradation** – If credentials expire, the node triggers `restart_process` to refresh tokens; if the control plane is down, nodes continue serving from the bucket indefinitely.

## Practical Implementation Examples

### Connecting from JavaScript

The WebSocket client connects to any node in the fleet—the underlying replication guarantees all nodes converge to the same state:

```javascript
// examples/wsclient/index.js
import { connect } from "https://deno.land/x/celld/ws_client.ts";

async function run() {
  // Connect to any node in the fleet
  const client = await connect("wss://my-fleet.example.com");

  // Write to shared storage
  await client.exec(`
    INSERT INTO kv (key, value) VALUES ('greeting', 'Hello, world!')
    ON CONFLICT(key) DO UPDATE SET value = excluded.value;
  `);

  // Read from any peer (request may be served by different node)
  const rows = await client.query("SELECT value FROM kv WHERE key = 'greeting'");
  console.log("Fetched:", rows[0].value);
}

run();

```

### Starting a Stateless Node

Launch a new node by pointing it to the shared bucket. No cluster registration or join protocol is required:

```bash

# Environment-based configuration

CELLD_NODE=my-node-id \
CELLD_BUCKET=s3://my-fleet-bucket \
CELLD_CONTROL_PLANE_URL=https://celld.dev \
celld start

```

- `CELLD_BUCKET` specifies the shared LTX log location (required)
- `CELLD_CONTROL_PLANE_URL` is optional; omitting it runs the node completely autonomously

The node immediately begins probing the bucket for log segments and joining the peer mesh via [`peer_probe.rs`](https://github.com/denoland/celld/blob/main/peer_probe.rs).

## Summary

- Celld achieves horizontal scaling without a traditional control plane or consensus by using **stateless nodes** that pull state from shared storage.
- The **optional control plane** ([`control_plane.rs`](https://github.com/denoland/celld/blob/main/control_plane.rs)) handles only bootstrap credentials via `ManagedRuntimeState`, not runtime coordination.
- **Peer discovery** operates via bi-directional probing ([`peer_probe.rs`](https://github.com/denoland/celld/blob/main/peer_probe.rs)) in a symmetric mesh with no leader.
- **LTX log replication** ([`ltx_repl.rs`](https://github.com/denoland/celld/blob/main/ltx_repl.rs), [`replication.rs`](https://github.com/denoland/celld/blob/main/replication.rs)) stores mutations in immutable, append-only segments within a shared bucket ([`bucket.rs`](https://github.com/denoland/celld/blob/main/bucket.rs)), allowing nodes to converge without consensus algorithms.
- New nodes join by simply reading the complete log history; departing nodes require no graceful shutdown coordination.

## Frequently Asked Questions

### Does celld require a control plane to operate?

No. The managed control plane is optional and only used during startup for credential provisioning and environment configuration. As implemented in [`crates/celld/control_plane.rs`](https://github.com/denoland/celld/blob/main/crates/celld/control_plane.rs), nodes can run entirely autonomously by specifying only `CELLD_BUCKET`. If the control plane becomes unavailable after startup, nodes transition to `ManagedRuntimeState::ControlPlaneUnavailable` and continue serving traffic using bucket storage alone.

### How does celld ensure consistency without a consensus algorithm?

Celld uses an append-only **LTX log** stored in shared object storage. Because log entries are immutable, every node can pull and apply them independently in any order. The [`replication.rs`](https://github.com/denoland/celld/blob/main/replication.rs) module reconciles conflicts using **CRDT-style merge rules**, guaranteeing that all replicas eventually converge to the same state without requiring Paxos, Raft, or two-phase commit protocols.

### What happens when a new node joins the fleet?

Adding capacity requires no coordination. A new node simply launches with the same `CELLD_BUCKET` URL, pulls the complete log history from the bucket via `Bucket::get`, joins the peer mesh through [`peer_probe.rs`](https://github.com/denoland/celld/blob/main/peer_probe.rs), and begins serving traffic immediately. There is no "join consensus" or cluster membership change protocol.

### How does the system handle node failures or network partitions?

Node failures require no failover logic. When a node departs, the remaining peers stop receiving its probes via the `probe` loop in [`peer_probe.rs`](https://github.com/denoland/celld/blob/main/peer_probe.rs) and remove it from their `PeerMap`. During network partitions, nodes continue writing to their local LTX logs and upload segments to the shared bucket; when connectivity returns, all nodes synchronize missed segments independently, relying on the immutability of the log to resolve conflicts.