How to Use pullOnce and pushOnce Helpers for Sync Rounds in Cloudflare Computer

The pullOnce and pushOnce helpers in packages/rpc/src/sync-driver.ts execute single-round synchronization between a local Database and a remote SyncRPC interface, where pullOnce applies remote changes locally and pushOnce uploads pending local changes while staging missing blobs on the remote.

The sync layer inside cloudflare/computer provides deterministic, state-machine-driven replication between container-side and Durable Object (DO) side databases. These pure functions manage cursor advancement, batch processing, and blob deduplication without owning timers or external retry loops. Understanding how to invoke these helpers directly allows you to build custom sync loops that operate on specific triggers rather than the default polling schedule.

Core Sync Helpers Overview

The sync driver exposes three public entry points for orchestrating replication rounds:

• pullOnce – Defined at lines 99‑105 of sync-driver.ts, this helper pulls every change produced by the remote since the last successful pull, applies those changes to the local Database, and advances the local fetch cursor.
• pushOnce – Defined at lines 91‑104 of the same file, this helper pushes every local change not yet sent to the remote, stages missing blob bytes via the RPC interface, and advances the local push cursor.
• tick – Defined at lines 107‑115, this executes a full sync round by calling pullOnce followed immediately by pushOnce. The pull‑then‑push order is intentional to prevent newly pulled remote writes from being immediately re-pushed.

All three helpers return promises that resolve with statistics about the work performed, making them suitable for both event-driven and polling-based architectures.

How pullOnce Processes Remote Changes

When you invoke pullOnce, the driver performs a multi-stage ingestion pipeline that bounds memory usage through batching.

Watermark Reconciliation and Cursor Initialization

Before pulling, the driver may invoke reconcileWatermarks (see sync-driver.ts, lines ≈ 350‑375) to ensure both sides start from a consistent baseline. If the remote’s log is shorter than the local view, cursors reset to 0 to prevent divergence.

Batch Processing with Bounded Memory

The pullOnce implementation reads the current fetch cursor via readFetchCursor, then calls remote.fetchChanges({ after }) to obtain a stream of ChangeEntry objects alongside the remote’s current cursor and applied push cursor.

Processing occurs in batches of PULL_BATCH_SIZE (256 entries) to limit heap consumption. For each batch, the driver:

1. Collects hashes of file chunks that might be missing locally.
2. Invokes remote.hasObjects to deduplicate transfer requirements.
3. Fetches missing blobs via remote.fetchObjects and stores them using stageBlob.
4. Applies the batch atomically with applyChanges.
5. Advances the local fetch cursor via writeFetchCursorIfAhead.

If the remote’s applied push cursor lags behind the local pushRev, or if the remote’s current cursor is behind the local after cursor, the driver resets the divergent cursor(s) and retries once (lines ≈ 57‑66) before returning.

How pushOnce Uploads Local Changes

The pushOnce helper handles the inverse direction, coalescing local mutations into efficient batches for transmission.

Batch Construction and Coalescing

pushOnce first reads the local push cursor via readPushCursor. It then calls pushBatch repeatedly until the batch reports status: "complete". Each batch respects two limits:

• Entry limit: PULL_BATCH_SIZE (256 changes)
• Byte budget: 4 MiB maximum

Inside pushBatch, the driver calls coalesceChanges (implemented in packages/dofs/src/sync/coalesce.ts) to group adjacent local modifications, reducing RPC overhead.

Blob Staging and Transmission

For each prepared batch, the execution flow is:

1. Determine which blob hashes the remote already possesses via remote.hasObjects.
2. Stream missing blobs to the remote using remote.pushObjects.
3. Transmit the coalesced entry stream via remote.push.
4. Update the local push cursor with writePushCursor upon successful acknowledgment.

pushOnce returns the total number of entries pushed once all batches complete successfully.

Executing a Full Sync Tick

For most applications, the tick helper provides the correct abstraction for a complete synchronization cycle. Located at lines 107‑115 of sync-driver.ts, tick executes:

const pulled = await pullOnce(db, remote);
const pushed = await pushOnce(db, remote);
return { pulled, pushed };

This ordering guarantees that remote changes are integrated into the local database before local changes are calculated for upload, preventing the "echo" problem where newly downloaded writes would otherwise be pushed back to their origin.

Practical Implementation Examples

The following patterns demonstrate how to integrate these helpers into a TypeScript application using the Cloudflare Computer SDK structure.

Basic Import and Setup

Import the helpers from the RPC package and obtain typed Database and SyncRPC instances:

import { pullOnce, pushOnce, tick } from "@cloudflare/rpc";
import type { Database } from "@cloudflare/dofs";
import type { SyncRPC } from "@cloudflare/rpc";

const db: Database = /* your local VFS instance */;
const remote: SyncRPC = /* DO-side RPC stub */;

Single-Direction Rounds

Execute isolated pull or push rounds when you need fine-grained control over sync direction:

async function runPull() {
  const result = await pullOnce(db, remote);
  console.log(`Pulled ${result.applied} entries`);
}

async function runPush() {
  const pushed = await pushOnce(db, remote);
  console.log(`Pushed ${pushed} entries`);
}

Full Synchronization Tick

Use tick for standard bidirectional synchronization:

async function syncTick() {
  const { pulled, pushed } = await tick(db, remote);
  console.log(`Sync complete: ${pulled.applied} pulled, ${pushed} pushed`);
}

Polling Loop Integration

Production deployments often wrap tick in a simple polling loop. The helpers are pure, so you control the scheduling:

setInterval(() => {
  syncTick().catch(err => console.error("Sync error:", err));
}, 5_000); // Execute every 5 seconds

Advanced Batch Control

While pullOnce and pushOnce use fixed batch sizes suitable for general workloads, memory-constrained environments may require tighter control. The driver exports lower-level pullBatch and pushBatch functions that accept explicit budgets:

import { pullBatch, type PullBatchOptions } from "@cloudflare/rpc";

const options: PullBatchOptions = {
  backend: "default",
  targetCursor: undefined, // Pull to latest
  budget: { 
    maxEntries: 128,      // Half the default batch size
    maxBytes: 2 * 1024 * 1024 // 2 MiB instead of 4 MiB
  }
};

const result = await pullBatch(db, remote, options);
console.log(`Batch status: ${result.status}`);

This technique is particularly useful when integrating with the FUSE driver implementation shown in packages/computerd/src/fuse/driver.ts, where synchronous latency requirements demand predictable memory bounds.

Summary

• pullOnce and pushOnce in packages/rpc/src/sync-driver.ts provide pure, stateless primitives for single-round database synchronization.
• Batching defaults to 256 entries (PULL_BATCH_SIZE) with a 4 MiB byte budget for pushes, preventing unbounded memory growth during large transfers.
• Watermark reconciliation (lines ≈ 350‑375) automatically corrects cursor divergences by resetting to baseline zero when the remote log is truncated.
• tick orchestrates the correct pull‑then‑push ordering to avoid re-uploading freshly downloaded remote changes.
• Lower-level batch functions allow custom memory and entry budgets for specialized environments like containerized FUSE filesystems.

Frequently Asked Questions

What is the difference between calling tick versus calling pullOnce and pushOnce separately?

tick is a convenience wrapper that enforces the correct execution order (pull followed by push) and returns a unified result object. Calling the helpers separately allows you to insert logic between the pull and push phases, such as user confirmation dialogs or intermediate data validation, but requires you to manually ensure you do not push before pulling, which could cause write echoes.

How does the sync driver handle cursor mismatches between the local database and remote?

When pullOnce detects that the remote’s applied push cursor is behind the local pushRev, or that the remote’s current cursor is behind the local fetch cursor, it invokes the retry logic at lines ≈ 57‑66 of sync-driver.ts. The driver resets the divergent cursor(s) to 0 and executes the batch operation once more. If the second attempt also fails, the promise rejects, signaling that manual watermark reconciliation or conflict resolution may be necessary.

Can I use pullOnce and pushOnce with a custom batch size to reduce memory usage?

Yes. While the high-level helpers use a fixed PULL_BATCH_SIZE of 256 entries and a 4 MiB byte budget, you can import pullBatch and pushBatch directly from @cloudflare/rpc. These functions accept a budget parameter allowing you to specify maxEntries and maxBytes values appropriate for your container’s memory constraints, as demonstrated in the container-side FUSE implementation.

Where can I find the protocol specification for the watermarks and cursors used by these helpers?

The high-level sync protocol, including semantics for fetch cursors, push cursors, and watermark reconciliation, is documented in docs/02_sync_protocol.md within the repository. The Cap’n Proto interface definitions for SyncRPC are located in packages/rpc/src/interface.ts, which defines the fetchChanges, hasObjects, and pushObjects methods invoked by the driver.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}