How to Implement Durable Pending-Sync Retries with SyncRetryScheduler

The Cloudflare Computer SDK enables durable pending-sync retries by persisting SyncRetryIntent objects through a pluggable SyncRetryScheduler interface that survives Durable Object restarts, automatically applying exponential backoff until success or exhaustion.

The Computer SDK provides a built-in mechanism for retrying failed post-command pull operations within a Durable Object. By implementing the SyncRetryScheduler interface and passing it to the Workspace constructor, you ensure that sync failures are retried reliably across DO invocations with configurable exponential backoff. This article explains the lifecycle of durable retries and provides a complete implementation guide based on the source code in packages/computer/src/workspace.ts.

Core Components of the Retry System

Three primary concepts work together to enable durable retries:

• SyncRetryIntent: A plain object defined at lines 47-51 of workspace.ts that records the backend identifier, current attempt count, and the earliest timestamp (notBefore) when the next retry may execute.
• SyncRetryScheduler: An interface (lines 59-63) that your Durable Object must implement, providing get, schedule, and clear operations to persist a single intent per backend.
• WorkspaceOptions.retryScheduler: A constructor option (lines 118-123) that accepts your scheduler instance. When provided, the workspace automatically stores intents and manages retry timing.

The Durable Retry Lifecycle

The retry mechanism follows a strict lifecycle from failure detection to exhaustion or success.

Detecting Post-Command Pull Failures

When a command finishes, the CommandExecutor—created in the #shellFor method—invokes onPullPending if the subsequent pull operation fails. This callback triggers the scheduling logic.

// From workspace.ts lines 885-889
const shell = new CommandExecutor(
    handle.rpc.shell,
    {
      push: () => this.push(id),
      pull: () => this.pull(id),
      onPullPending: () => this.#schedulePendingSync(id), // Schedules intent
    },
    this.#observer,
);

Scheduling the First Retry Intent

The private method #schedulePendingSync (lines 511-557) checks the durable scheduler for existing intents. If none exist, it creates a new SyncRetryIntent with attempt count 1 and persists it.

async #schedulePendingSync(id: string): Promise<void> {
    const scheduler = this.#retryScheduler;
    if (scheduler === undefined) return;
    await this.#serialize(id, async (resolvedId) => {
        if (resolvedId === undefined || (await scheduler.get(resolvedId)) !== undefined) return;
        await scheduler.schedule(this.#retryIntent(resolvedId, 1));
    });
}

Exponential Backoff Calculation

The #retryIntent helper (lines 660-666) calculates delays using exponential growth capped by maxDelayMs:

#retryIntent(backend: string, attempt: number): SyncRetryIntent {
    const delay = Math.min(
        this.#retryMaxDelayMs,
        this.#retryInitialDelayMs * 2 ** Math.max(0, attempt - 1),
    );
    return { backend, attempt, notBefore: this.#now() + delay };
}

Executing Retries via Durable Object Alarms

Your Durable Object must use its alarm handler (or custom timer) to invoke retryPendingSync. This method reads the stored intent, validates the attempt count, and either completes the sync or reschedules with an incremented attempt number.

async retryPendingSync(id?: string): Promise<WorkspaceRetryPendingSyncResult> {
    const resolvedId = this.#resolveBackendId(id);
    const scheduler = this.#retryScheduler!;
    const intent = await scheduler.get(resolvedId);
    if (!intent) return { status: "idle", backend: resolvedId };
    if (intent.attempt > this.#retryMaxAttempts) {
        return { status: "exhausted", backend: resolvedId, attempt: intent.attempt, error: "pending sync retry attempts exhausted" };
    }
    try {
        const result = await this.#pullResolved(resolvedId);
        await scheduler.clear(resolvedId);
        return { status: "complete", backend: resolvedId, applied: result.applied, skipped: result.skipped };
    } catch (error) {
        const next = this.#retryIntent(resolvedId, intent.attempt + 1);
        await scheduler.schedule(next);
        return { status: "pending", ...next, error: safeErrorMessage(error) };
    }
}

Handling Exhaustion

When intent.attempt exceeds maxAttempts, the method returns { status: "exhausted", ... }. Your host code should detect this status and surface the error to users or trigger alternative recovery workflows.

Implementing a Durable Scheduler

A scheduler implementation requires only a key-value store that persists across DO invocations. The following example uses the Durable Object storage API:

import { SyncRetryScheduler, SyncRetryIntent } from "@cloudflare/computer";

class DurableSyncRetryScheduler implements SyncRetryScheduler {
    constructor(private readonly storage: DurableObjectStorage) {}

    async get(backend: string): Promise<SyncRetryIntent | undefined> {
        return this.storage.get<SyncRetryIntent>(backend);
    }

    async schedule(intent: SyncRetryIntent): Promise<void> {
        await this.storage.put(intent.backend, intent);
    }

    async clear(backend: string): Promise<void> {
        await this.storage.delete(backend);
    }
}

Complete Integration Example

The following example demonstrates initializing a Workspace with a durable scheduler and wiring it to the Durable Object alarm:

import { Workspace } from "@cloudflare/computer";

export class MyDurableObject {
    private workspace: Workspace;
    private scheduler: DurableSyncRetryScheduler;

    constructor(private state: DurableObjectState) {
        this.scheduler = new DurableSyncRetryScheduler(state.storage);
        this.workspace = new Workspace({
            storage: state.storage,
            backends: [/* your backends */],
            retryScheduler: this.scheduler,
            retry: { 
                initialDelayMs: 2_000, 
                maxDelayMs: 30_000, 
                maxAttempts: 4 
            },
        });
    }

    async fetch(request: Request): Promise<Response> {
        const handle = await this.workspace.runtime.exec("build");
        const result = await handle.result();
        
        // If sync is pending, schedule the alarm
        if (result.sync === "pending") {
            await this.scheduleAlarmIfNeeded();
        }
        
        return new Response("OK");
    }

    async alarm(): Promise<void> {
        // Retry all pending backends
        for (const backend of this.workspace.backends) {
            const result = await this.workspace.retryPendingSync(backend.id);
            if (result.status === "pending") {
                await this.scheduleAlarmIfNeeded();
            }
        }
    }

    private async scheduleAlarmIfNeeded(): Promise<void> {
        for (const backend of this.workspace.backends) {
            const intent = await this.scheduler.get(backend.id);
            if (intent) {
                const delay = intent.notBefore - Date.now();
                await this.state.storage.setAlarm(Date.now() + Math.max(0, delay));
                break;
            }
        }
    }
}

Testing and Validation

The repository's test suite in packages/computer/src/retry.test.ts validates the retry semantics:

• Intent persistence: Verifies that SyncRetryIntent objects survive Durable Object restarts and contain correct notBefore timestamps calculated from the mocked clock.
• Idempotency: Confirms that concurrent failures do not spawn multiple intents for the same backend.
• Backoff behavior: Asserts exponential growth, delay capping at maxDelayMs, and proper exhaustion handling when attempts exceed maxAttempts.
• Resource cleanup: Ensures failed retry envelopes and RPC handles are properly disposed.

These tests demonstrate that the scheduler contract is enforced through TypeScript and that the workspace correctly manages the transition from pending to complete or exhausted states.

Summary

• SyncRetryScheduler is a required interface for durable retries, implemented using any persistent key-value store such as DurableObjectStorage.
• The Computer SDK automatically schedules retries via #schedulePendingSync when post-command pulls fail, using exponential backoff capped by maxDelayMs.
• retryPendingSync handles the execution logic, validating attempt counts against maxAttempts and either clearing successful intents or rescheduling failed ones.
• The system guarantees at-least-once delivery of sync attempts while preventing duplicate retries for the same backend through idempotent scheduling.

Frequently Asked Questions

How does the retry mechanism persist across Durable Object restarts?

The SyncRetryScheduler interface abstracts persistence through its get, schedule, and clear methods. When you implement this interface using DurableObjectStorage, the SyncRetryIntent objects are written to disk via storage.put(). Since Durable Object storage survives restarts, the intent remains available when the alarm triggers and calls retryPendingSync, allowing the workspace to resume exactly where it left off.

What is the exponential backoff formula used by the Computer SDK?

The SDK calculates delay using Math.min(maxDelayMs, initialDelayMs * 2 ** Math.max(0, attempt - 1)). This means the first retry waits initialDelayMs, the second waits initialDelayMs * 2, the third waits initialDelayMs * 4, and so on until reaching maxDelayMs. The formula is implemented in the #retryIntent private method at lines 660-666 of workspace.ts.

How do you handle exhausted retry attempts in a production environment?

When retryPendingSync returns { status: "exhausted" }, the intent remains stored unless you explicitly clear it. You should check for this status in your alarm handler or fetch method, then either alert the user, log the failure to external monitoring, or trigger a manual reconciliation workflow. The exhaustion state is permanent until you call scheduler.clear(backend) or the storage is manually purged.

Can multiple concurrent failures create duplicate retry intents for the same backend?

No. The #schedulePendingSync method uses this.#serialize to ensure atomicity when checking for existing intents. If an intent already exists for a backend, the method returns early without creating a duplicate. This idempotency guarantee prevents redundant alarms and redundant pull attempts when multiple commands fail simultaneously against the same backend.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}