How Cloudflare Computer Synchronizes Changes Between Durable Objects and Execution Environments

Cloudflare Computer uses a bidirectional sync driver in packages/rpc/src/sync-driver.ts that reconciles watermarks, pulls remote changes in bounded batches, and pushes local updates to maintain a consistent virtual filesystem across edge-based Durable Objects and containerized execution environments.

The cloudflare/computer repository implements a distributed runtime requiring strict state consistency between edge-resident Durable Objects and ephemeral container environments. To achieve this, the platform employs a deterministic synchronization protocol that treats the Durable Object as the source of truth while allowing bidirectional mutation through a strict watermark-based reconciliation process.

The Bidirectional Sync Driver Architecture

The synchronization logic lives in packages/rpc/src/sync-driver.ts, which defines the bidirectional sync driver mediating between a local SQLite-backed database (@cloudflare/dofs) and a remote RPC stub implementing the SyncRPC interface. This driver operates on a pull-then-push cycle orchestrated by the tick function, ensuring that execution environments maintain an eventually consistent view of the virtual filesystem.

The Cap’n Web RPC contract (docs/08_capnweb_interface.md) formalizes the communication surface between the Durable Object and the container. The interface exposes six critical methods: fetchChanges, hasObjects, fetchObjects, push, pushObjects, and watermarks. Both the Durable Object stub and the container stub implement this contract, enabling transparent streaming of filesystem changes with version-aware semantics.

Step‑by‑Step Synchronization Protocol

1. Reconcile Watermarks on (Re)connect

Before any data transfer occurs, the driver invokes reconcileWatermarks (lines 370–375 in sync-driver.ts) to align the local and remote state vectors. This function compares the remote’s watermarks against the local fetch and push cursors:

• If the Durable Object’s log is shorter than the container remembers, the fetch cursor resets to revision 0.
• If the Durable Object has not seen pushes that the container believes it transmitted, the push cursor resets to revision 0.

This defensive check guarantees that fresh connections recover gracefully from crashes or network partitions without manual intervention.

2. Pull Changes from the Durable Object

The pullOnce function (or its batched variant pullBatch) requests entries produced by the Durable Object since the last successful fetch cursor. The implementation streams changes in bounded batches of 256 entries (PULL_BATCH_SIZE at lines 81–84) to maintain predictable memory usage.

For each entry, the driver determines missing blob data through remote.hasObjects and retrieves it via remote.fetchObjects, staging blobs locally with stageBlob. After applying changes idempotently through applyChanges, the driver advances the fetch cursor. If watermark divergence is detected during this process, the driver automatically resets cursors and retries (lines 57–70).

3. Push Local Changes to the Durable Object

When propagating local mutations, pushOnce (or pushBatch) gathers entries since the last push cursor (readPushCursor). The protocol first identifies which object blobs the remote lacks using remote.hasObjects, then streams those blobs via remote.pushObjects. Subsequently, it transmits the change entries through remote.push and validates acknowledgment with assertAppliedPushCursor before persisting the updated push cursor locally (lines 188–186).

4. Orchestration with the Tick Loop

The high-level tick function (lines 106–117) orchestrates a full synchronization round by strictly ordering operations: pull first, then push. This sequencing matters because pulling remote updates before examining local dirty sets prevents unnecessary re-pushes of data that the Durable Object already contains. The function returns statistics on pulled and pushed entries, allowing callers to monitor sync health.

Resource Management and Resilience Guarantees

The synchronization layer enforces three operational invariants:

• Exactly-once semantics: Watermark cursors (fetch and push) ensure each change applies exactly once, even across process restarts.
• Bounded resource usage: Fixed-size batches (256 entries) and configurable byte budgets keep memory and bandwidth consumption predictable.
• Resilience to divergence: Automatic cursor resets and retry mechanisms (implemented in pullOnce and pushOnce) recover from transient mismatches without manual operator intervention.

Implementation Example

The following TypeScript example demonstrates initializing the sync driver and executing a reconciliation cycle:

import { openDatabase } from "@cloudflare/dofs";
import { SyncRPC } from "@cloudflare/computer-rpc";
import { tick, reconcileWatermarks } from "@cloudflare/computer-rpc";

// 1. Open the local SQLite database on the container
const db = await openDatabase("local.db");

// 2. Create a Remote RPC stub pointing to the Durable Object
const remote: SyncRPC = await getDurableObjectStub();

// 3. Reconcile watermarks on every (re)connect
await reconcileWatermarks(db, remote);

// 4. Run a synchronization tick – pulls then pushes
const { pulled, pushed } = await tick(db, remote);
console.log(`Pulled ${pulled.applied} entries, pushed ${pushed} entries`);

For advanced use cases requiring manual budget control, use the low-level batch APIs:

import { pullBatch, pushBatch } from "@cloudflare/computer-rpc";

// Pull with memory constraints
const pullResult = await pullBatch(db, remote, {
  budget: { maxEntries: 200, maxBytes: 2 * 1024 * 1024 },
});
if (pullResult.status === "pending") {
  // Handle partial result and resume later
}

// Push with similar constraints
const pushResult = await pushBatch(db, remote, {
  budget: { maxEntries: 200, maxBytes: 2 * 1024 * 1024 },
});

Summary

• The bidirectional sync driver in packages/rpc/src/sync-driver.ts maintains consistency between Durable Objects and execution environments through a strict pull-then-push protocol.
• Watermark reconciliation (reconcileWatermarks) ensures crash recovery by resetting cursors when logs diverge.
• Bounded batching (PULL_BATCH_SIZE = 256) and idempotent change application (applyChanges) protect against memory exhaustion and duplicate processing.
• The Cap’n Web RPC contract (docs/08_capnweb_interface.md) defines the interface methods (fetchChanges, pushObjects, etc.) that enable transparent state streaming.
• Exactly-once semantics and automatic retry logic make the system resilient to network partitions and process restarts.

Frequently Asked Questions

What is the role of watermarks in Cloudflare Computer’s synchronization?

Watermarks act as vector clocks tracking the fetch and push cursors between the Durable Object and the execution environment. They ensure exactly-once processing by marking which changes have been successfully transmitted and applied. When reconnecting, reconcileWatermarks compares these cursors to detect gaps or rollbacks, automatically resetting to revision 0 if the remote log diverges from local expectations.

How does the sync driver handle large filesystem changes without exhausting memory?

The driver implements bounded batching through PULL_BATCH_SIZE (set to 256 entries) and configurable byte budgets in pullBatch and pushBatch. By streaming changes in fixed-size chunks rather than loading entire filesystem snapshots, the system maintains predictable memory usage regardless of data volume.

Why does the tick function pull changes before pushing them?

The pull-then-push ordering prevents circular data transmission. By absorbing remote updates first through applyChanges, the driver ensures that any modifications already present in the Durable Object are merged into the local state before examining the container’s dirty set. This eliminates redundant re-pushes of data that originated from the remote side.

Where is the RPC interface between the Durable Object and container defined?

The Cap’n Web RPC contract is specified in docs/08_capnweb_interface.md and implemented in packages/rpc/src/interface.ts. It defines six core methods—fetchChanges, hasObjects, fetchObjects, push, pushObjects, and watermarks—that both the Durable Object stub and the container stub implement to enable version-aware, bidirectional state synchronization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}