OpenSEO Rank Tracking System Architecture: A Complete Technical Breakdown

OpenSEO's rank tracking system is a workflow orchestrated, server-side architecture built on Cloudflare Workers that schedules SERP checks, executes them via DataForSEO, persists snapshots to SQLite/Postgres, and exposes results through type-safe server functions.

The every-app/open-seo repository implements a production-grade keyword rank tracking system designed for reliability, cost control, and horizontal scalability. This architecture separates concerns across distinct layers—from scheduling and credit validation to execution and persistence—while leveraging Cloudflare Workers Workflows for durable, step-wise execution.

Core Architecture Layers

Workflow Orchestration Layer

The RankCheckWorkflow.ts file serves as the central entry point for all rank-check operations. This workflow class coordinates the entire lifecycle of a rank tracking run.

// Pseudocode representing the workflow structure
class RankCheckWorkflow {
  async run({ runId, configId, trigger, keywordIds }) {
    // 1. Validate credits via Autumn
    // 2. Prepare keyword list
    // 3. Execute live or queued check
    // 4. Finalize run on completion
  }
}

The workflow performs three critical validation steps before execution:

• Input validation — confirms config exists and user has access
• Credit checking — queries Autumn billing service to enforce usage limits
• Cost estimation — calculates expected spend based on keyword count and device combinations

Run Preparation and Cost Control

The prepareRankCheckKeywords function (lines 52-99 in RankCheckWorkflow.ts) filters the keyword list and prepares execution parameters. This layer applies the optional keywordIds filter when users trigger partial checks, then estimates costs before any external API calls occur.

Cost enforcement happens through the Autumn integration at src/server/billing/autumn.ts. The system blocks execution if credits are insufficient, preventing accidental overspend on DataForSEO usage.

Execution Paths: Live vs. Queued

The architecture supports two distinct execution strategies implemented in rankCheckPaths.ts:

Live Execution (runLiveCheck)

• Use case: Manual triggers requiring immediate results
• Behavior: Synchronous calls to DataForSEO's "rank-check" endpoint
• Throughput: Limited by DataForSEO rate limits and Worker CPU time

Queued Execution (runQueuedCheck)

• Use case: Scheduled checks with large keyword volumes
• Behavior: Asynchronous task submission with polling loop
• Fallback: Automatically promotes failed queued tasks to live calls

// Task expansion and batch processing
const tasks = expandToTaskInputs(keywords, devices);
// Submits to DataForSEO task queue
const posted = await postTasksToDataForSEO(tasks);
// Polls until completion or timeout
const results = await collectQueuedRound(posted.taskIds);

The expandToTaskInputs function generates all keyword-device combinations, while checkBatchLive and collectQueuedRound handle the DataForSEO API surface (lines 55-115 in rankCheckPaths.ts).

Database Persistence with Atomic Batches

Rank snapshots are stored through the RankTrackingRepository class, which provides Drizzle ORM abstractions over SQLite (local) or Postgres (production).

Key repository methods include:

• insertSnapshots — bulk inserts rank data with run attribution
• updateRun — tracks progress counts and status transitions
• getSnapshotsForRun — retrieves historical results for analysis

Each batch runs inside a pgStep — Cloudflare Workers' durable execution primitive — ensuring atomic writes and automatic retry semantics. This design allows partial progress to survive transient failures without corrupting run state.

The snapshot schema captures:

• runId and keywordId for lineage
• Device type (desktop/mobile)
• SERP position and ranked URL
• Detected SERP features (featured snippets, knowledge panels, etc.)

Scheduling and Automation

The scheduledRankChecks.ts service implements cron-style automation:

// Claims configs due for checking based on frequency settings
const dueConfigs = await claimDueRankConfigs();
// Creates runs and triggers workflow for each
await Promise.all(dueConfigs.map(c => createAndTriggerRun(c)));

This service respects user-configured check frequencies and integrates with the same workflow orchestrator used for manual triggers, ensuring consistent behavior across execution modes.

Public API Surface

Frontend interactions flow through type-safe server functions in src/serverFunctions/rank-tracking.ts:

// Trigger a manual rank check from the front-end
await triggerRankCheck({
  configId: "cfg_01",
  keywordIds: ["kw_123", "kw_456"], // optional filter
});

// Get the latest run information (status, counts, etc.)
const latestRun = await getLatestRankRun({
  configId: "cfg_01",
});

// Retrieve the most recent rank snapshots for a config
const results = await getLatestRankResults({
  configId: "cfg_01",
  comparePeriod: "30d",
});

These functions delegate to RankTrackingService.ts for business logic and RankTrackingRepository.ts for data access, maintaining clean separation between API contracts and implementation details.

Data Flow Summary

A complete rank check follows this sequence:

1. Trigger — User or scheduler calls triggerRankCheck or scheduled job claims due config
2. Run creation — RankTrackingRepository.tryCreateRun inserts pending run record
3. Workflow activation — RankCheckWorkflow.run validates credits via Autumn
4. Keyword preparation — prepareRankCheckKeywords loads and filters target keywords
5. Execution routing — Manual → runLiveCheck, Scheduled → runQueuedCheck
6. Batch processing — Cloudflare pgStep executes DataForSEO calls with retry logic
7. Snapshot persistence — Results written to database after each batch completes
8. Finalization — finalizeRankCheckRun aggregates counts, updates status to "completed", and refreshes lastCheckedAt

Technology Stack

Component	Technology	Purpose
Workflow engine	Cloudflare Workers Workflows	Durable, step-wise execution with automatic retries
Database ORM	Drizzle ORM	Type-safe SQL for SQLite/Postgres portability
SERP provider	DataForSEO	External API for Google/Bing rank data
Billing enforcement	Autumn	Credit checking and usage metering
Runtime	Cloudflare Workers	Edge-deployed, autoscaling compute

Summary

• OpenSEO's rank tracking architecture separates concerns across workflow orchestration, execution strategies, database persistence, and public API layers
• Cloudflare Workers Workflows provide the foundation for reliable, retry-safe execution without managing infrastructure
• Two execution paths (live and queued) optimize for latency versus throughput depending on trigger type
• Atomic batch processing via pgStep ensures data consistency even during partial failures
• Autumn integration enforces cost controls before any billable external API calls
• Drizzle ORM abstractions enable database portability between SQLite (development) and Postgres (production)

Frequently Asked Questions

What is OpenSEO's rank tracking system built on?

OpenSEO's rank tracking system is built on Cloudflare Workers Workflows for orchestration, Drizzle ORM for database access, DataForSEO as the SERP data provider, and Autumn for billing enforcement. The entire stack runs server-side with no long-running processes to manage.

How does OpenSEO handle large keyword volumes?

For large volumes, the system uses queued execution (runQueuedCheck in rankCheckPaths.ts) which submits tasks to DataForSEO's asynchronous queue and polls for results. This avoids Worker CPU time limits. Failed queued tasks automatically fall back to live calls to maximize completion rates.

Where does OpenSEO store rank tracking data?

Rank data persists in SQLite or Postgres via Drizzle ORM, depending on environment. The RankTrackingRepository.ts file implements all data access, storing individual snapshots with run attribution, device type, position, URL, and SERP features. Batches write atomically through Cloudflare's pgStep primitive.

How does OpenSEO prevent overspending on SERP checks?

The prepareRankCheckKeywords function in RankCheckWorkflow.ts calculates expected costs before execution. The Autumn billing service (src/server/billing/autumn.ts) validates sufficient credits exist. If credits are insufficient, the workflow aborts before any DataForSEO API calls, preventing accidental charges.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}