How OpenSEO Tracks Keyword Positions: A 4-Stage Pipeline Explained

OpenSEO tracks keyword positions through a four-stage pipeline—trigger, workflow dispatch, DataForSEO SERP lookup, and persistence—storing results in a rank_snapshots table for historical comparison and reporting.

If you're building or integrating SEO rank tracking into your workflow, understanding how OpenSEO implements this feature provides a complete blueprint. The every-app/open-seo repository implements position tracking via a server-side workflow that orchestrates third-party SERP data, cost-aware scheduling, and durable storage. This article breaks down the exact mechanics, file locations, and code patterns used in production.

The Four Stages of OpenSEO Rank Tracking

Stage 1: Trigger — Initiating a Rank Check

Every position tracking run begins with a trigger call. When a user clicks "Run now" in the UI or an automation fires, the triggerCheck server function validates permissions, checks available credits, and creates a run record.

// Client-side trigger call
import { createClient } from '@/client';

const client = createClient();
await client.rankTracking.triggerCheck({
  projectId: 'a0c1e2f3-4b5d-6e7f-8g9h-0123456789ab',
  configId: 'd1e2f3g4-5h6i-7j8k-9l0m-1234567890ab',
});

The RankTrackingService.ts handles this in src/server/features/rank-tracking/services/RankTrackingService.ts. It performs three critical tasks:

• Creates a rank_check_runs row to track execution state
• Validates that the user has sufficient credits (calling estimateRankCheckCredits from src/shared/rank-tracking.ts)
• Hands off to the workflow engine with runLiveCheck or runQueuedCheck

Stage 2: Workflow Dispatch — Live vs. Queued Execution

The workflow engine in src/server/workflows/rankCheckPaths.ts orchestrates two execution modes:

• runLiveCheck — Immediate synchronous execution for on-demand checks
• runQueuedCheck — Asynchronous task-queue execution for scheduled or bulk operations

This file contains the core resilience logic: retries with exponential backoff, polling intervals for queued tasks, and automatic fallback from queued to live if deadlines approach. The workflow engine ensures that a stuck task_get poll doesn't block indefinitely—instead, it degrades gracefully to the live endpoint to maintain data freshness.

Stage 3: DataForSEO SERP Lookup — Fetching Raw Positions

For each keyword/device pair (desktop or mobile), OpenSEO calls the DataForSEO SERP API through two distinct patterns:

Live mode (checkBatchLive):

// Inside rankCheckPaths.ts
await client.serp.rankCheck({
  keyword,
  location_code,
  language_code,
  device,
  // ... other params
});

Queued mode (runQueuedCheck):

1. Post tasks via client.serp.rankCheckTaskPost
2. Poll task_get endpoint until results ready
3. Fallback to live if timeout threshold exceeded

The DataForSEO client wrapper lives in src/server/lib/dataforseo/client.ts. It normalizes responses across both modes so downstream code receives a consistent shape regardless of how the data was fetched.

Stage 4: Persistence & Reporting — Storing Snapshots

Returned SERP data transforms into snapshot rows and inserts via RankTrackingRepository.insertSnapshots in src/server/features/rank-tracking/repositories/RankTrackingRepository.ts.

Each rank_snapshots row contains:

Column	Purpose
runId	Links to the parent execution
trackingKeywordId	References the keyword configuration
keyword	Raw search term text
device	desktop or mobile
position	Numeric SERP rank (null if not in top 100)
url	Landing page URL from DataForSEO
serpFeatures	JSON array of feature types (featured snippet, video carousel, etc.)

The schema definition appears in both src/db/app.schema.ts and src/db/pg/app.schema.ts for database portability.

Cost-Aware Scheduling and Credit Estimation

OpenSEO prevents surprise bills through upfront cost estimation. The estimateRankCheckCredits function in src/shared/rank-tracking.ts calculates:

costCredits = keywordsCount × devicesCount × costPerKeyword
costUsd = costCredits / AUTUMN_SEO_DATA_CREDITS_PER_USD

If estimated cost exceeds a threshold, the system returns rankCheckCostApprovalError and requires explicit user confirmation before proceeding.

For recurring schedules (daily/weekly/monthly), computeNextCheckAt (same file) computes the next execution timestamp. This prevents schedule drift—even if a Tuesday run delays to Wednesday, the following run still targets the next scheduled Tuesday slot, maintaining consistent reporting periods.

Retrieving and Comparing Positions

The UI consumes tracked positions through the getLatestResults service method:

const latest = await client.rankTracking.getLatestResults({
  projectId,
  configId,
  comparePeriod: '30d', // '1d' | '7d' | '30d' | '90d'
});

Response structure includes position movement by comparing current snapshots against historical baseline:

{
  trackingKeywordId: '...',
  keyword: 'open source seo tools',
  desktop: {
    position: 3,
    rankingUrl: 'https://example.com/seo-tools',
    serpFeatures: ['featured_snippet', 'people_also_ask']
  },
  mobile: {
    position: 7, // Often differs from desktop
    rankingUrl: 'https://example.com/seo-tools',
    serpFeatures: []
  }
}

End-to-End Integration Example

// 1. Trigger a check
await fetch('/api/rank-tracking/trigger', {
  method: 'POST',
  body: JSON.stringify({ projectId, configId })
});

// 2. Workflow executes (live or queued mode automatically)
//    - DataForSEO API calls
//    - Result transformation
//    - Snapshot insertion

// 3. Query latest positions with historical comparison
const results = await fetch('/api/rank-tracking/latest', {
  method: 'POST',
  body: JSON.stringify({ projectId, configId, comparePeriod: '7d' })
});

Summary

• Four-stage pipeline: trigger → workflow dispatch → DataForSEO lookup → snapshot persistence
• Two execution modes: live (instant) and queued (cost-optimized with polling)
• Cost protection: upfront estimation with estimateRankCheckCredits and approval gates
• Durable storage: rank_snapshots table with full historical tracking and SERP feature capture
• Device-level granularity: separate desktop and mobile positions per keyword
• Drift-free scheduling: computeNextCheckAt maintains consistent recurring intervals

Frequently Asked Questions

How does OpenSEO handle API failures during rank checking?

The workflow engine in src/server/workflows/rankCheckPaths.ts implements automatic retries with exponential backoff and live fallback for queued tasks. If a queued task_get poll exceeds the timeout threshold, the system degrades to the live endpoint to ensure data delivery rather than failing silently.

What SERP features does OpenSEO capture beyond position rankings?

Each snapshot stores a serpFeatures JSON array capturing elements like featured snippets, video carousels, People Also Ask boxes, and local packs. These extract from the raw DataForSEO response and enable SERP feature penetration analysis alongside pure rank tracking.

Can I estimate costs before running a large keyword batch?

Yes—call estimateRankCheckCredits via the client or API (implemented in src/shared/rank-tracking.ts). The calculation multiplies keyword count by device count by per-keyword credit cost, returning both credit and USD estimates. Large checks trigger rankCheckCostApprovalError until explicitly approved.

How does OpenSEO ensure consistent scheduling for daily/weekly/monthly checks?

The computeNextCheckAt function calculates next execution based on calendar period boundaries rather than simple interval delays. This prevents drift: if a Monday daily check runs late at 11 PM, the next check still targets Tuesday at the configured time—not 23 hours later.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}