How Scheduled Rank Tracking Executes Keyword Checks in Open-SEO

Open-SEO handles scheduled keyword checks through a cron-driven pipeline that atomically claims due tracker configurations and launches a Cloudflare Workflow, ensuring time-of-day consistency and automatic retry on failure.

Scheduled rank tracking is the backbone of any SEO monitoring platform. In the every-app/open-seo repository, the system orchestrates periodic SERP checks through a carefully designed three-phase pipeline that balances cost efficiency, schedule accuracy, and fault tolerance.

The Three-Phase Rank Check Pipeline

Open-SEO divides scheduled keyword checks into distinct phases: cron selection, workflow initiation, and queued execution. Each phase uses specific services and repositories to maintain clean separation of concerns.

Phase 1: Cron Identifies and Claims Due Trackers

A Cloudflare Worker with a scheduled handler triggers runScheduledRankChecks in src/server/features/rank-tracking/services/scheduledRankChecks.ts (lines 36-84). This function queries RankTrackingRepository.getDueConfigsWithOrganization to find every tracker whose next_check_at has passed.

For each due configuration, the cron performs three critical operations:

1. Compute anchor-preserving next check time with computeNextCheckAt(interval, observedNextCheckAt) — this preserves the original time-of-day and prevents drift across intervals
2. Validate organization credits for hosted plans (lines 88-115)
3. Claim the slot atomically via RankTrackingRepository.claimDueConfig to eliminate race conditions between concurrent cron invocations

The computeNextCheckAt function lives in src/shared/rank-tracking.ts and ensures that a daily check at 9:00 AM stays at 9:00 AM rather than creeping forward with each execution delay.

Phase 2: Workflow Launch with Guard Protection

Once claimed, the configuration passes to beginRankCheckRun (invoked from line 71 of scheduledRankChecks.ts). This guard function in src/server/features/rank-tracking/services/rankCheckRunGuards.ts:

• Creates a RankTrackingRun database row with status pending
• Generates a unique runId
• Invokes the Cloudflare Workflow via env.RANK_CHECK_WORKFLOW

The payload explicitly sets trigger: "scheduled" — this flag determines the execution path in the next phase.

Phase 3: Workflow Chooses Queued vs. Live Check

RankCheckWorkflow in src/server/workflows/RankCheckWorkflow.ts (lines 46-58) unpacks the payload and verifies the config remains active. It then calls prepareRankCheckKeywords to load keywords and estimate costs.

The critical branch occurs at lines 53-60:

// Workflow core – chooses queued vs live based on trigger
if (trigger === "scheduled") {
  // cheaper task‑queue endpoint
  queueStats = await runQueuedCheck(step, checkContext);
} else {
  await runLiveCheck(step, checkContext);
}

runQueuedCheck submits batches of up to MAX_TASKS_PER_POST keywords to DataForSEO's task queue API — significantly cheaper than the live endpoint used for on-demand checks. After batch completion, the workflow finalizes statistics, updates lastCheckedAt, and clears any temporary lastSkipReason.

Schedule Integrity and Failure Handling

The design intentionally separates schedule advancement from workflow completion to prevent drift.

Next-Check-Time Computation Happens in Cron

The cron computes nextCheckAt before starting the workflow (lines 106-112 of scheduledRankChecks.ts). This guarantees:

• Failed workflows do not push the schedule forward
• The next cron tick naturally retries overdue trackers
• Time-of-day anchors remain stable regardless of execution latency

// Compute the next scheduled time (preserves time-of-day anchor)
export function computeNextCheckAt(
  interval: ScheduledRankTrackingInterval,
  previousNextCheckAt?: string | null,
): string {
  // ...implementation from src/shared/rank-tracking.ts (lines 78‑115)
}

Conflict Recovery on Duplicate Runs

If beginRankCheckRun detects an in-progress run, the cron restores the original schedule via claimDueConfig with swapped observedNextCheckAt and nextCheckAt values (lines 166-172).

Credit-Based Cost Control

Before any API calls, prepareRankCheckKeywords estimates queued costs through estimateRankCheckCredits. The workflow aborts with INSUFFICIENT_CREDITS if the hosted plan lacks balance (lines 88-99 of RankCheckWorkflow.ts).

When credits are insufficient:

• The config's lastSkipReason becomes "insufficient_credits"
• The UI displays this status for user visibility
• The schedule still advances (unlike execution failures) since this is a billing constraint, not a transient error

Cron Resilience Limits

The scheduled handler respects operational guardrails:

• SCHEDULED_TASK_UNIT_BUDGET — caps work per invocation to stay within Cloudflare Worker limits
• TICK_DEADLINE_MS — soft deadline to ensure clean shutdown before hard timeouts

These constraints appear at lines 94-101 of scheduledRankChecks.ts.

Summary

• Cron-driven selection: runScheduledRankChecks queries due configs, validates credits, and claims slots atomically
• Workflow abstraction: RankCheckWorkflow handles both scheduled (queued) and live execution paths through the same generic interface
• Cost optimization: Scheduled checks use DataForSEO's cheaper task-queue endpoint; credit validation occurs before any API spend
• Schedule durability: computeNextCheckAt preserves time-of-day anchors, and failed runs do not advance the schedule — the next cron tick retries automatically
• Race-condition safety: claimDueConfig with optimistic locking prevents duplicate workflow launches across concurrent cron instances

Frequently Asked Questions

What prevents the same rank tracker from running twice simultaneously?

The RankTrackingRepository.claimDueConfig method performs an atomic database update that swaps next_check_at only when the row still matches the expected observedNextCheckAt. If another cron instance claims it first, the update returns zero rows and the current invocation skips that tracker. Additionally, beginRankCheckRun in rankCheckRunGuards.ts verifies no pending run exists before creating a new workflow instance.

How does Open-SEO handle missed scheduled checks?

Missed checks are automatically retried on the next cron tick because the schedule only advances after successful workflow initiation. If a workflow fails or the cron hits its budget/deadline, next_check_at remains unchanged. The tracker appears in getDueConfigsWithOrganization results again until successfully claimed and started.

Why do scheduled checks use a different DataForSEO endpoint?

Scheduled checks invoke runQueuedCheck which uses DataForSEO's task queue API, while on-demand checks use runLiveCheck with the live endpoint. The queued API offers lower per-request pricing at the cost of asynchronous results — acceptable for automated monitoring but not for immediate user-facing queries. The trigger: "scheduled" payload flag determines this branch in RankCheckWorkflow.ts.

What happens when an organization runs out of credits mid-check?

Credit validation occurs twice: first in the cron for hosted plan verification, then in prepareRankCheckKeywords which estimates total cost before any API requests. If credits are insufficient, the workflow aborts with INSUFFICIENT_CREDITS, sets lastSkipReason for UI display, and skips the DataForSEO calls entirely. This prevents partial spend and failed individual keyword requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}