How OpenSEO Implements Rank Tracking with Cloudflare Workflows: A Technical Architecture Guide

OpenSEO implements rank tracking as a Cloudflare Workflow that queries pending configurations from PostgreSQL, constructs live SERP requests for the DataForSEO API, and persists results back to the database using isolated execution steps with automatic retry logic.

OpenSEO is an open-source SEO platform built entirely on Cloudflare's edge infrastructure. According to the every-app/open-seo source code, the rank tracking module leverages Cloudflare Workflows to execute scheduled keyword position checks without requiring persistent background worker processes or server containers.

Architecture Overview

The rank tracking system follows a pull-based cron architecture orchestrated by Cloudflare's native Workflow product. When a user configures rank tracking for a domain, the system stores the configuration in rank_tracking_configs and calculates a next_run_at timestamp. A Cron trigger registered in wrangler.jsonc periodically invokes the RankTrackingWorkflow.ts entry point, which processes all due configurations in a single workflow execution.

Each workflow instance runs through five distinct steps:

1. Query configurations where next_run_at <= now()
2. Build DataForSEO live-rank payloads
3. Execute API calls to DataForSEO
4. Insert results into rank_tracking_results
5. Update next_run_at timestamps for the next cycle

Data Schema and Configuration

Rank tracking configurations are defined in src/types/schemas/rank-tracking.ts. The schema captures the target domain, geographic location (location_code), device type, language preferences, and the keyword list to monitor.

// src/types/schemas/rank-tracking.ts
interface RankTrackingConfig {
  id: string;
  domain: string;
  location_code: number;
  language_code: string;
  devices: 'desktop' | 'mobile';
  keywords: string[];
  next_run_at: Date;
  schedule: 'hourly' | 'daily' | 'weekly';
}

The next_run_at field acts as a cursor for the cron-based polling mechanism. When the workflow completes successfully, it increments this timestamp based on the schedule interval, ensuring the next invocation processes only fresh jobs.

Workflow Implementation

The core orchestration logic resides in src/server/workflows/RankTrackingWorkflow.ts. This file exports a Cloudflare Workflow class where each method represents an atomic step in the execution chain.

Step 1: Loading Pending Configurations

The workflow begins by querying the database for configurations due for execution. Using Prisma (or Drizzle ORM as indicated in the source), the workflow fetches rows where next_run_at has passed.

// src/server/workflows/RankTrackingWorkflow.ts
async function loadPendingConfigs(db: PrismaClient) {
  return await db.rankTrackingConfig.findMany({
    where: {
      next_run_at: {
        lte: new Date()
      }
    },
    take: 100 // Batch limit per workflow invocation
  });
}

Step 2: Building DataForSEO Requests

For each keyword in the configuration, the workflow constructs a payload using helper functions defined in src/shared/rank-tracking.ts. The buildLiveRankPayload function maps OpenSEO's internal configuration format to DataForSEO's live SERP API structure.

// src/shared/rank-tracking.ts
export function buildLiveRankPayload(
  config: RankTrackingConfig,
  keyword: string
): DataForSEO.LiveRankRequest {
  return {
    keyword,
    location_code: config.location_code,
    language_code: config.language_code,
    device: config.devices,
    se_type: 'organic'
  };
}

export function estimateRankTrackerCost(keywordCount: number): number {
  return keywordCount * 0.002; // Cost per live SERP check
}

Step 3: Executing Live Rank Checks

The workflow issues POST requests to the DataForSEO live endpoint. Authentication uses the DATAFORSEO_API_KEY environment variable defined in worker-configuration.d.ts.

// src/server/workflows/RankTrackingWorkflow.ts
const response = await fetch('https://api.dataforseo.com/v3/serp/google/organic/live/', {
  method: 'POST',
  headers: {
    'Authorization': `Basic ${btoa(env.DATAFORSEO_API_KEY + ':')}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify(payload)
});

Step 4: Persisting Results

Upon receiving the SERP data, the workflow parses the JSON response to extract position, URL, and SERP features. It then bulk-inserts records into rank_tracking_results.

// src/server/workflows/RankTrackingWorkflow.ts
const results = data.tasks[0].result[0].items.map((item, index) => ({
  config_id: config.id,
  keyword: config.keywords[index],
  position: item.rank_absolute,
  url: item.url,
  serp_features: item.se_results_count,
  checked_at: new Date()
}));

await db.rankTrackingResult.createMany({ data: results });

Integration with DataForSEO API

OpenSEO uses DataForSEO's Live Rank endpoint (/v3/serp/google/organic/live/) rather than the queue-based system. This provides synchronous results within the workflow execution, eliminating the need for webhook callbacks or secondary polling workflows. The estimateRankTrackerCost function in src/shared/rank-tracking.ts calculates API expenses before execution, allowing the workflow to skip batches that would exceed budget limits.

Scheduling and Cron Triggers

The workflow is triggered via Cloudflare's Cron Triggers defined in wrangler.jsonc. A dedicated Worker entry point invokes the Workflow instance:

// wrangler.jsonc
{
  "triggers": {
    "crons": ["0 * * * *"] // Every hour
  }
}

When the Cron fires, the Worker instantiates RankTrackingWorkflow and passes the database binding and environment variables. Because Cloudflare Workflows execute each step in an isolated context with automatic retries, transient failures in the DataForSEO API or database connections are handled transparently without manual intervention.

Summary

• Cloudflare Workflows provide the execution engine for OpenSEO's rank tracking, offering built-in retry logic and step isolation.
• Configuration persistence uses PostgreSQL tables (rank_tracking_configs, rank_tracking_results) with next_run_at timestamps for cursor-based polling.
• DataForSEO integration leverages the live SERP endpoint via src/shared/rank-tracking.ts helper functions like buildLiveRankPayload.
• Cron scheduling is configured in wrangler.jsonc and triggers the RankTrackingWorkflow.ts entry point on customizable intervals.
• Cost estimation occurs before API calls via estimateRankTrackerCost, preventing budget overruns on high-volume keyword lists.

Frequently Asked Questions

How does OpenSEO handle API rate limits when checking thousands of keywords?

The workflow processes configurations in batches (typically 100 per invocation) and implements the estimateRankTrackerCost function from src/shared/rank-tracking.ts to pre-calculate expenses. If the estimated cost exceeds available credits, the workflow skips the batch and updates the next_run_at timestamp to retry in the next cycle. Additionally, Cloudflare Workflows' built-in retry policies with exponential backoff handle transient 429 responses from DataForSEO automatically.

What happens if the DataForSEO API request fails during workflow execution?

Because each API call runs inside a distinct workflow step, Cloudflare Workflows automatically retry failed steps according to the configured retry policy (default is 3 attempts with exponential backoff). If all retries exhaust, the workflow marks that specific configuration's next_run_at to a future timestamp, allowing the error to be logged without blocking other rank tracking jobs. The isolation ensures one failed keyword check doesn't crash the entire batch.

How is the scheduling interval configured for rank tracking jobs?

Users select intervals (hourly, daily, or weekly) when creating a rank tracking configuration in rank_tracking_configs. The workflow reads this schedule field after successfully completing a check, then calculates the next timestamp relative to the current time. The Cron trigger defined in wrangler.jsonc typically runs every hour to pick up any due jobs, but the actual frequency per domain respects the user-selected interval stored in the database.

Can the workflow handle multiple search engines or just Google?

Currently, the buildLiveRankPayload function in src/shared/rank-tracking.ts hardcodes se_type: 'organic' and targets google/organic/live in the DataForSEO endpoint URL. While the schema supports extensibility, the current implementation focuses exclusively on Google SERPs. To support Bing or other engines, developers would need to modify the endpoint construction logic in RankTrackingWorkflow.ts and add a search_engine column to the rank_tracking_configs schema.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}