How RankCheckWorkflow Handles Scheduled Rank Tracking with Retry Logic

The RankCheckWorkflow routes scheduled triggers through runQueuedCheck and wraps every phase in pgStep, providing configurable automatic retries, transactional safety, and strict single-attempt finalization to guarantee consistent state.

The RankCheckWorkflow in every-app/open-seo is the core orchestrator for both manual and scheduled rank-tracking runs. When handling scheduled rank tracking with retry logic, it delegates keyword batches to DataForSEO's cheaper task queue and guards each phase with the pgStep helper. This architecture isolates transient failures, prevents duplicate writes, and ensures users are never billed for doomed runs.

Trigger Routing and the Queued Check Path

In src/server/workflows/RankCheckWorkflow.ts, the workflow first inspects the trigger parameter to decide which DataForSEO endpoint to use. When trigger === "scheduled", the workflow calls runQueuedCheck from src/server/workflows/rankCheckPaths.ts instead of runLiveCheck.

if (trigger === "scheduled") {
  // Use DataForSEO's task-queue (≈30% of live cost)
  queueStats = await runQueuedCheck(step, checkContext);
} else {
  // Immediate live endpoint for manual checks
  await runLiveCheck(step, checkContext);
}

runQueuedCheck submits keyword batches to DataForSEO's asynchronous task queue. This path reduces API spend by roughly seventy percent and allows the system to spread ingestion over time, making it ideal for cron-driven schedules. Manual checks continue to use runLiveCheck for instant results.

pgStep: Retry Logic and Transaction Wrapper

Every logical phase executes through pgStep (src/server/workflows/pgStep.ts), a thin wrapper around a Cloudflare Workers Workflow step. pgStep accepts a step instance, a descriptive name, a retry configuration object, and a callback. It executes the callback inside a Postgres transaction—or a D1-compatible shim—retries on transient failures according to the per-step policy, and enforces step-specific timeouts.

A typical guarded step looks like this:

const configCheck = await pgStep(
  step,
  "check-active",
  { retries: { limit: 0, delay: "1 second" } },
  async () => {
    const cfg = await RankTrackingRepository.getConfigById({ configId, projectId });
    return { isActive: cfg?.isActive ?? false };
  },
);

The retry policy is defined per step. Long-running operations like SERP fetching can tolerate multiple retries, while finalization steps use SINGLE_ATTEMPT_STEP_CONFIG to avoid duplicate writes. If a callback fails with a retryable error—such as a temporary database deadlock—pgStep automatically re-executes it up to the configured retries.limit, propagating consistent errors back to the workflow.

Batched Processing and Step-Level Resilience

For scheduled runs, runQueuedCheck breaks the keyword list into batches. Each batch is processed inside its own pgStep, inheriting the same retry semantics.

If a batch encounters a transient failure—for example, a network hiccup to DataForSEO—the step retries without requiring custom back-off logic in the caller. After all batches complete or a terminal exception is raised, the workflow proceeds to finalizeRankCheckRun. This layered approach keeps partial failures isolated to individual batches rather than invalidating the entire run.

Credit and Guard Safeguards Before Execution

Before any DataForSEO API calls begin, the workflow estimates credit cost using estimateRankCheckCredits from src/shared/rank-tracking.ts. For scheduled runs, the queued pricing model ("queued") is applied, ensuring the user has sufficient credits before the long-running task starts.

If credits are insufficient, an AppError with code INSUFFICIENT_CREDITS is thrown. The config is then updated with lastSkipReason: "insufficient_credits" so the UI can explain why the scheduled check was skipped. This guard prevents runs from consuming queue slots and API budget when they are destined to fail.

Failure Handling and Exactly-Once Finalization

When any step throws an exception, the outer try…catch block in RankCheckWorkflow.ts captures it:

try {
  // … main workflow body …
} catch (error) {
  console.error(`Rank check ${runId} failed:`, error);
  await pgStep(step, "mark-failed", SINGLE_ATTEMPT_STEP_CONFIG, async () =>
    markRankCheckRunFailed({ runId, configId, projectId, billingCustomer, error })
  );
  throw error;
}

Transient errors—such as temporary DB issues—are retried automatically by pgStep. Non-retryable errors—for instance, when a stale-cleanup job has already marked the run as failed—are raised as NonRetryableError early in prepareRankCheckKeywords according to utilities in src/server/features/rank-tracking/services/rankCheckRunGuards.ts. These abort the workflow immediately without further retries.

The finalization step is wrapped with SINGLE_ATTEMPT_STEP_CONFIG (retries: { limit: 0 }). This guarantees the run's status is written exactly once, preventing race conditions with the cron handler that may schedule the next run.

Summary

• The RankCheckWorkflow discriminates between scheduled and manual triggers, routing scheduled runs to runQueuedCheck for cheaper, asynchronous processing.
• pgStep wraps every phase in a transactional, retry-aware Cloudflare Workers step with configurable retry limits and timeouts.
• Batched keyword processing inside runQueuedCheck inherits automatic retry logic, isolating transient network failures to individual batches.
• Credit verification using estimateRankCheckCredits aborts under-funded scheduled runs before API consumption begins.
• Finalization via finalizeRankCheckRun and failure marking via markRankCheckRunFailed both use SINGLE_ATTEMPT_STEP_CONFIG to enforce exactly-once state updates.

Frequently Asked Questions

What is the difference between runQueuedCheck and runLiveCheck in RankCheckWorkflow?

runQueuedCheck submits keyword batches to DataForSEO's asynchronous task queue and is used when the trigger is scheduled. It costs roughly thirty percent of the live endpoint and spreads work over time. runLiveCheck hits the instant SERP endpoint directly and is reserved for manual checks that require immediate results.

How does pgStep handle retries for scheduled rank tracking steps?

pgStep re-executes the callback when it encounters retryable failures like temporary database deadlocks or transient network issues, up to a per-step retries.limit. Each step can define its own retry policy. For example, batched SERP fetches may retry multiple times, while finalization steps use SINGLE_ATTEMPT_STEP_CONFIG with zero retries to avoid duplicate writes.

What happens if a scheduled rank check runs out of credits?

Before calling DataForSEO, the workflow invokes estimateRankCheckCredits with the "queued" pricing model. If the account lacks sufficient credits, an AppError with code INSUFFICIENT_CREDITS is thrown. The config's lastSkipReason is set to "insufficient_credits", and the workflow skips the run without consuming API resources.

Why does the finalization step use a single-attempt configuration?

The finalization step uses SINGLE_ATTEMPT_STEP_CONFIG (retries: { limit: 0 }) to guarantee that the run status is persisted exactly once. This prevents race conditions where a cron handler schedules the next run while the previous workflow is still retrying its final write, which could otherwise lead to duplicate or conflicting state updates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}