# How OpenSEO Tracks Keyword Positions: A 4-Stage Pipeline Explained

> Discover how OpenSEO tracks keyword positions using its efficient four-stage pipeline. Learn how data is processed and stored for historical analysis and reporting.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: deep-dive
- Published: 2026-08-20

---

**OpenSEO tracks keyword positions through a four-stage pipeline—trigger, workflow dispatch, DataForSEO SERP lookup, and persistence—storing results in a `rank_snapshots` table for historical comparison and reporting.**

If you're building or integrating **SEO rank tracking** into your workflow, understanding how OpenSEO implements this feature provides a complete blueprint. The `every-app/open-seo` repository implements position tracking via a server-side workflow that orchestrates third-party SERP data, cost-aware scheduling, and durable storage. This article breaks down the exact mechanics, file locations, and code patterns used in production.

## The Four Stages of OpenSEO Rank Tracking

### Stage 1: Trigger — Initiating a Rank Check

Every position tracking run begins with a **trigger call**. When a user clicks "Run now" in the UI or an automation fires, the `triggerCheck` server function validates permissions, checks available credits, and creates a run record.

```typescript
// Client-side trigger call
import { createClient } from '@/client';

const client = createClient();
await client.rankTracking.triggerCheck({
  projectId: 'a0c1e2f3-4b5d-6e7f-8g9h-0123456789ab',
  configId: 'd1e2f3g4-5h6i-7j8k-9l0m-1234567890ab',
});

```

The [`RankTrackingService.ts`](https://github.com/every-app/open-seo/blob/main/RankTrackingService.ts) handles this in [`src/server/features/rank-tracking/services/RankTrackingService.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/rank-tracking/services/RankTrackingService.ts). It performs three critical tasks:

- Creates a `rank_check_runs` row to track execution state
- Validates that the user has sufficient credits (calling `estimateRankCheckCredits` from [`src/shared/rank-tracking.ts`](https://github.com/every-app/open-seo/blob/main/src/shared/rank-tracking.ts))
- Hands off to the workflow engine with `runLiveCheck` or `runQueuedCheck`

### Stage 2: Workflow Dispatch — Live vs. Queued Execution

The workflow engine in [`src/server/workflows/rankCheckPaths.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/rankCheckPaths.ts) orchestrates **two execution modes**:

- **`runLiveCheck`** — Immediate synchronous execution for on-demand checks
- **`runQueuedCheck`** — Asynchronous task-queue execution for scheduled or bulk operations

This file contains the core resilience logic: **retries with exponential backoff**, polling intervals for queued tasks, and automatic fallback from queued to live if deadlines approach. The workflow engine ensures that a stuck `task_get` poll doesn't block indefinitely—instead, it degrades gracefully to the live endpoint to maintain data freshness.

### Stage 3: DataForSEO SERP Lookup — Fetching Raw Positions

For each **keyword/device pair** (desktop or mobile), OpenSEO calls the **DataForSEO SERP API** through two distinct patterns:

**Live mode** (`checkBatchLive`):

```typescript
// Inside rankCheckPaths.ts
await client.serp.rankCheck({
  keyword,
  location_code,
  language_code,
  device,
  // ... other params
});

```

**Queued mode** (`runQueuedCheck`):
1. Post tasks via `client.serp.rankCheckTaskPost`
2. Poll `task_get` endpoint until results ready
3. Fallback to live if timeout threshold exceeded

The DataForSEO client wrapper lives in [`src/server/lib/dataforseo/client.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/client.ts). It normalizes responses across both modes so downstream code receives a consistent shape regardless of how the data was fetched.

### Stage 4: Persistence & Reporting — Storing Snapshots

Returned SERP data transforms into **snapshot rows** and inserts via `RankTrackingRepository.insertSnapshots` in [`src/server/features/rank-tracking/repositories/RankTrackingRepository.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/rank-tracking/repositories/RankTrackingRepository.ts).

Each `rank_snapshots` row contains:

| Column | Purpose |
|--------|---------|
| `runId` | Links to the parent execution |
| `trackingKeywordId` | References the keyword configuration |
| `keyword` | Raw search term text |
| `device` | `desktop` or `mobile` |
| `position` | Numeric SERP rank (null if not in top 100) |
| `url` | Landing page URL from DataForSEO |
| `serpFeatures` | JSON array of feature types (featured snippet, video carousel, etc.) |

The schema definition appears in both [`src/db/app.schema.ts`](https://github.com/every-app/open-seo/blob/main/src/db/app.schema.ts) and [`src/db/pg/app.schema.ts`](https://github.com/every-app/open-seo/blob/main/src/db/pg/app.schema.ts) for database portability.

## Cost-Aware Scheduling and Credit Estimation

OpenSEO prevents surprise bills through **upfront cost estimation**. The `estimateRankCheckCredits` function in [`src/shared/rank-tracking.ts`](https://github.com/every-app/open-seo/blob/main/src/shared/rank-tracking.ts) calculates:

```typescript
costCredits = keywordsCount × devicesCount × costPerKeyword
costUsd = costCredits / AUTUMN_SEO_DATA_CREDITS_PER_USD

```

If estimated cost exceeds a threshold, the system returns `rankCheckCostApprovalError` and requires explicit user confirmation before proceeding.

For **recurring schedules** (daily/weekly/monthly), `computeNextCheckAt` (same file) computes the next execution timestamp. This prevents schedule drift—even if a Tuesday run delays to Wednesday, the following run still targets the next scheduled Tuesday slot, maintaining consistent reporting periods.

## Retrieving and Comparing Positions

The UI consumes tracked positions through the `getLatestResults` service method:

```typescript
const latest = await client.rankTracking.getLatestResults({
  projectId,
  configId,
  comparePeriod: '30d', // '1d' | '7d' | '30d' | '90d'
});

```

Response structure includes **position movement** by comparing current snapshots against historical baseline:

```typescript
{
  trackingKeywordId: '...',
  keyword: 'open source seo tools',
  desktop: {
    position: 3,
    rankingUrl: 'https://example.com/seo-tools',
    serpFeatures: ['featured_snippet', 'people_also_ask']
  },
  mobile: {
    position: 7, // Often differs from desktop
    rankingUrl: 'https://example.com/seo-tools',
    serpFeatures: []
  }
}

```

## End-to-End Integration Example

```typescript
// 1. Trigger a check
await fetch('/api/rank-tracking/trigger', {
  method: 'POST',
  body: JSON.stringify({ projectId, configId })
});

// 2. Workflow executes (live or queued mode automatically)
//    - DataForSEO API calls
//    - Result transformation
//    - Snapshot insertion

// 3. Query latest positions with historical comparison
const results = await fetch('/api/rank-tracking/latest', {
  method: 'POST',
  body: JSON.stringify({ projectId, configId, comparePeriod: '7d' })
});

```

## Summary

- **Four-stage pipeline**: trigger → workflow dispatch → DataForSEO lookup → snapshot persistence
- **Two execution modes**: live (instant) and queued (cost-optimized with polling)
- **Cost protection**: upfront estimation with `estimateRankCheckCredits` and approval gates
- **Durable storage**: `rank_snapshots` table with full historical tracking and SERP feature capture
- **Device-level granularity**: separate desktop and mobile positions per keyword
- **Drift-free scheduling**: `computeNextCheckAt` maintains consistent recurring intervals

## Frequently Asked Questions

### How does OpenSEO handle API failures during rank checking?

The workflow engine in [`src/server/workflows/rankCheckPaths.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/rankCheckPaths.ts) implements **automatic retries with exponential backoff** and **live fallback** for queued tasks. If a queued `task_get` poll exceeds the timeout threshold, the system degrades to the live endpoint to ensure data delivery rather than failing silently.

### What SERP features does OpenSEO capture beyond position rankings?

Each snapshot stores a **`serpFeatures`** JSON array capturing elements like featured snippets, video carousels, People Also Ask boxes, and local packs. These extract from the raw DataForSEO response and enable **SERP feature penetration analysis** alongside pure rank tracking.

### Can I estimate costs before running a large keyword batch?

Yes—call `estimateRankCheckCredits` via the client or API (implemented in [`src/shared/rank-tracking.ts`](https://github.com/every-app/open-seo/blob/main/src/shared/rank-tracking.ts)). The calculation multiplies keyword count by device count by per-keyword credit cost, returning both credit and USD estimates. Large checks trigger `rankCheckCostApprovalError` until explicitly approved.

### How does OpenSEO ensure consistent scheduling for daily/weekly/monthly checks?

The `computeNextCheckAt` function calculates next execution based on **calendar period boundaries** rather than simple interval delays. This prevents drift: if a Monday daily check runs late at 11 PM, the next check still targets Tuesday at the configured time—not 23 hours later.