# OpenSEO Rank Tracking System Architecture: A Complete Technical Breakdown

> Explore the OpenSEO rank tracking system architecture. Discover how Cloudflare Workers schedule SERP checks, use DataForSEO, store data in SQLite/Postgres, and expose results via type-safe server functions.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: architecture
- Published: 2026-08-14

---

**OpenSEO's rank tracking system is a workflow orchestrated, server-side architecture built on Cloudflare Workers that schedules SERP checks, executes them via DataForSEO, persists snapshots to SQLite/Postgres, and exposes results through type-safe server functions.**

The **every-app/open-seo** repository implements a production-grade keyword rank tracking system designed for reliability, cost control, and horizontal scalability. This architecture separates concerns across distinct layers—from scheduling and credit validation to execution and persistence—while leveraging Cloudflare Workers Workflows for durable, step-wise execution.

## Core Architecture Layers

### Workflow Orchestration Layer

The [`RankCheckWorkflow.ts`](https://github.com/every-app/open-seo/blob/main/RankCheckWorkflow.ts) file serves as the **central entry point** for all rank-check operations. This workflow class coordinates the entire lifecycle of a rank tracking run.

```typescript
// Pseudocode representing the workflow structure
class RankCheckWorkflow {
  async run({ runId, configId, trigger, keywordIds }) {
    // 1. Validate credits via Autumn
    // 2. Prepare keyword list
    // 3. Execute live or queued check
    // 4. Finalize run on completion
  }
}

```

The workflow performs three critical validation steps before execution:

- **Input validation** — confirms config exists and user has access
- **Credit checking** — queries Autumn billing service to enforce usage limits
- **Cost estimation** — calculates expected spend based on keyword count and device combinations

### Run Preparation and Cost Control

The `prepareRankCheckKeywords` function (lines 52-99 in [`RankCheckWorkflow.ts`](https://github.com/every-app/open-seo/blob/main/RankCheckWorkflow.ts)) filters the keyword list and prepares execution parameters. This layer applies the optional `keywordIds` filter when users trigger partial checks, then estimates costs before any external API calls occur.

Cost enforcement happens through the **Autumn integration** at [`src/server/billing/autumn.ts`](https://github.com/every-app/open-seo/blob/main/src/server/billing/autumn.ts). The system blocks execution if credits are insufficient, preventing accidental overspend on DataForSEO usage.

## Execution Paths: Live vs. Queued

The architecture supports two distinct execution strategies implemented in [`rankCheckPaths.ts`](https://github.com/every-app/open-seo/blob/main/rankCheckPaths.ts):

### Live Execution (`runLiveCheck`)

- **Use case**: Manual triggers requiring immediate results
- **Behavior**: Synchronous calls to DataForSEO's "rank-check" endpoint
- **Throughput**: Limited by DataForSEO rate limits and Worker CPU time

### Queued Execution (`runQueuedCheck`)

- **Use case**: Scheduled checks with large keyword volumes
- **Behavior**: Asynchronous task submission with polling loop
- **Fallback**: Automatically promotes failed queued tasks to live calls

```typescript
// Task expansion and batch processing
const tasks = expandToTaskInputs(keywords, devices);
// Submits to DataForSEO task queue
const posted = await postTasksToDataForSEO(tasks);
// Polls until completion or timeout
const results = await collectQueuedRound(posted.taskIds);

```

The `expandToTaskInputs` function generates all keyword-device combinations, while `checkBatchLive` and `collectQueuedRound` handle the DataForSEO API surface (lines 55-115 in [`rankCheckPaths.ts`](https://github.com/every-app/open-seo/blob/main/rankCheckPaths.ts)).

## Database Persistence with Atomic Batches

Rank snapshots are stored through the `RankTrackingRepository` class, which provides **Drizzle ORM** abstractions over SQLite (local) or Postgres (production).

Key repository methods include:

- `insertSnapshots` — bulk inserts rank data with run attribution
- `updateRun` — tracks progress counts and status transitions
- `getSnapshotsForRun` — retrieves historical results for analysis

Each batch runs inside a `pgStep` — Cloudflare Workers' durable execution primitive — ensuring **atomic writes and automatic retry semantics**. This design allows partial progress to survive transient failures without corrupting run state.

The snapshot schema captures:

- `runId` and `keywordId` for lineage
- Device type (desktop/mobile)
- SERP position and ranked URL
- Detected SERP features (featured snippets, knowledge panels, etc.)

## Scheduling and Automation

The [`scheduledRankChecks.ts`](https://github.com/every-app/open-seo/blob/main/scheduledRankChecks.ts) service implements cron-style automation:

```typescript
// Claims configs due for checking based on frequency settings
const dueConfigs = await claimDueRankConfigs();
// Creates runs and triggers workflow for each
await Promise.all(dueConfigs.map(c => createAndTriggerRun(c)));

```

This service respects user-configured check frequencies and integrates with the same workflow orchestrator used for manual triggers, ensuring consistent behavior across execution modes.

## Public API Surface

Frontend interactions flow through type-safe server functions in [`src/serverFunctions/rank-tracking.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/rank-tracking.ts):

```typescript
// Trigger a manual rank check from the front-end
await triggerRankCheck({
  configId: "cfg_01",
  keywordIds: ["kw_123", "kw_456"], // optional filter
});

// Get the latest run information (status, counts, etc.)
const latestRun = await getLatestRankRun({
  configId: "cfg_01",
});

// Retrieve the most recent rank snapshots for a config
const results = await getLatestRankResults({
  configId: "cfg_01",
  comparePeriod: "30d",
});

```

These functions delegate to [`RankTrackingService.ts`](https://github.com/every-app/open-seo/blob/main/RankTrackingService.ts) for business logic and [`RankTrackingRepository.ts`](https://github.com/every-app/open-seo/blob/main/RankTrackingRepository.ts) for data access, maintaining clean separation between API contracts and implementation details.

## Data Flow Summary

A complete rank check follows this sequence:

1. **Trigger** — User or scheduler calls `triggerRankCheck` or scheduled job claims due config
2. **Run creation** — `RankTrackingRepository.tryCreateRun` inserts pending run record
3. **Workflow activation** — `RankCheckWorkflow.run` validates credits via Autumn
4. **Keyword preparation** — `prepareRankCheckKeywords` loads and filters target keywords
5. **Execution routing** — Manual → `runLiveCheck`, Scheduled → `runQueuedCheck`
6. **Batch processing** — Cloudflare `pgStep` executes DataForSEO calls with retry logic
7. **Snapshot persistence** — Results written to database after each batch completes
8. **Finalization** — `finalizeRankCheckRun` aggregates counts, updates status to "completed", and refreshes `lastCheckedAt`

## Technology Stack

| Component | Technology | Purpose |
|-----------|-----------|---------|
| Workflow engine | Cloudflare Workers Workflows | Durable, step-wise execution with automatic retries |
| Database ORM | Drizzle ORM | Type-safe SQL for SQLite/Postgres portability |
| SERP provider | DataForSEO | External API for Google/Bing rank data |
| Billing enforcement | Autumn | Credit checking and usage metering |
| Runtime | Cloudflare Workers | Edge-deployed, autoscaling compute |

## Summary

- **OpenSEO's rank tracking architecture** separates concerns across workflow orchestration, execution strategies, database persistence, and public API layers
- **Cloudflare Workers Workflows** provide the foundation for reliable, retry-safe execution without managing infrastructure
- **Two execution paths** (live and queued) optimize for latency versus throughput depending on trigger type
- **Atomic batch processing** via `pgStep` ensures data consistency even during partial failures
- **Autumn integration** enforces cost controls before any billable external API calls
- **Drizzle ORM abstractions** enable database portability between SQLite (development) and Postgres (production)

## Frequently Asked Questions

### What is OpenSEO's rank tracking system built on?

OpenSEO's rank tracking system is built on **Cloudflare Workers Workflows** for orchestration, **Drizzle ORM** for database access, **DataForSEO** as the SERP data provider, and **Autumn** for billing enforcement. The entire stack runs server-side with no long-running processes to manage.

### How does OpenSEO handle large keyword volumes?

For large volumes, the system uses **queued execution** (`runQueuedCheck` in [`rankCheckPaths.ts`](https://github.com/every-app/open-seo/blob/main/rankCheckPaths.ts)) which submits tasks to DataForSEO's asynchronous queue and polls for results. This avoids Worker CPU time limits. Failed queued tasks automatically fall back to live calls to maximize completion rates.

### Where does OpenSEO store rank tracking data?

Rank data persists in **SQLite or Postgres** via Drizzle ORM, depending on environment. The [`RankTrackingRepository.ts`](https://github.com/every-app/open-seo/blob/main/RankTrackingRepository.ts) file implements all data access, storing individual snapshots with run attribution, device type, position, URL, and SERP features. Batches write atomically through Cloudflare's `pgStep` primitive.

### How does OpenSEO prevent overspending on SERP checks?

The `prepareRankCheckKeywords` function in [`RankCheckWorkflow.ts`](https://github.com/every-app/open-seo/blob/main/RankCheckWorkflow.ts) calculates expected costs before execution. The **Autumn billing service** ([`src/server/billing/autumn.ts`](https://github.com/every-app/open-seo/blob/main/src/server/billing/autumn.ts)) validates sufficient credits exist. If credits are insufficient, the workflow aborts before any DataForSEO API calls, preventing accidental charges.