# How Open‑SEO's Rank Tracking Guard System Prevents API Over‑Fetching

> Open-SEO's rank tracking guard system stops API over-fetching using tick budgets, deadlines, and stale-run detection. Optimize your SERP API calls and prevent duplicates.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: internals
- Published: 2026-08-10

---

**Open‑SEO uses a multi‑layered guard system combining per‑tick budgets, wall‑clock deadlines, database‑level unique indexes, and stale‑run detection to cap SERP API calls and eliminate duplicate or runaway fetches.**

The `every-app/open-seo` repository implements a sophisticated **rank tracking guard system** that protects external SERP API quotas from over‑consumption. Rather than relying on a single check, the system layers five independent safeguards across the scheduler, database schema, and workflow coordination. This article breaks down exactly how each mechanism works, where it lives in the codebase, and how they interact to guarantee API‑call limits are never exceeded.

## Task‑Unit Budget Guard: Capping Per‑Tick Work

The scheduler's primary defense against burst traffic is a **hard limit on keyword‑device combinations** that can be initiated in a single tick.

In [`src/server/features/rank-tracking/services/scheduledRankChecks.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/rank-tracking/services/scheduledRankChecks.ts), the constant `SCHEDULED_TASK_UNIT_BUDGET = 1000` defines this ceiling. Before launching any run, the code calculates:

```typescript
const taskUnits = kwCount * devicesCount(config.devices);

```

If `unitsStarted + taskUnits` would breach the budget, the loop terminates with `stoppedByBudget = true`. This ensures that even a backlog of thousands of due configurations cannot trigger more than 1,000 units worth of API calls in one scheduler execution.

## Wall‑Clock Deadline: Preventing Worker Monopolization

Complementing the unit budget, a **time‑based killswitch** prevents any single tick from running indefinitely.

The same file declares `TICK_DEADLINE_MS = 3 * 60_000` (three minutes). At each iteration, the scheduler compares `Date.now()` against `deadline = tickStart + TICK_DEADLINE_MS`. Once exceeded, the loop exits regardless of remaining budget. This protects against:

- Slow database queries stalling the scheduler
- Unexpected await‑blocks extending execution
- Overlapping tick executions caused by cron drift

## Partial Unique Index: Database‑Level Exclusivity

The core coordination primitive is a **PostgreSQL partial unique index** that enforces single‑run semantics at the data layer.

In [`src/db/app.schema.ts`](https://github.com/every-app/open-seo/blob/main/src/db/app.schema.ts), the index is defined as:

```sql
CREATE UNIQUE INDEX rank_check_runs_config_id_active
ON rank_check_runs (config_id)
WHERE status IN ('pending','running');

```

This guarantees that for any given `config_id`, only one row may exist with `status = 'pending'` or `'running'`. Attempts to insert a duplicate fail with a constraint violation, which the application treats as an `"already_running"` signal. Unlike application‑level locking, this is **race‑condition proof** and survives process crashes.

## Stale‑Run Detection: Reclaiming Orphaned Slots

When the unique index blocks insertion, the system must distinguish between valid in‑progress runs and **zombie runs** whose workflows have died.

The logic resides in [`src/server/features/rank-tracking/services/rankCheckRunGuards.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/rank-tracking/services/rankCheckRunGuards.ts). When blocked, `beginRankCheckRun` fetches the blocking run via `getActiveRunForConfig`, then evaluates staleness through `getStaleRankCheckRunReason`:

```typescript
const staleReason = await getStaleRankCheckRunReason({
  run: blocker,
  runId: blocker.id,
  ageMs: Date.now() - new Date(blocker.startedAt).getTime(),
});

```

A run is stale if its workflow is missing, errored, terminated, or exceeds the startup grace window. When detected, `failRunIfActive(blocker.id, staleReason, blocker)` marks it failed—**releasing the unique‑index slot**—and the insertion retries.

## Paid‑Plan and Zero‑Keyword Skip Logic

Before any API‑bound work begins, the scheduler applies **business‑rule guards** to avoid wasteful operations.

In [`scheduledRankChecks.ts`](https://github.com/every-app/open-seo/blob/main/scheduledRankChecks.ts), the function `checkPaidPlan` validates the organization's subscription. Free‑tier configs are claimed with `lastSkipReason: "plan_required"` and never reach the SERP API. Similarly, configs with zero keywords receive `lastSkipReason: "no_keywords"`. These early exits populate `skippedFree` and `skippedNoKeywords` metrics for observability.

## End‑to‑End Execution Flow

The five guards operate in strict sequence during each scheduler tick:

1. **Select due configs** — Query returns configs where `next_check_at` ≤ now.
2. **Budget and deadline enforcement** — Loop aborts when `SCHEDULED_TASK_UNIT_BUDGET` or `TICK_DEADLINE_MS` is exhausted.
3. **Plan and keyword validation** — Free or empty configs are claimed and skipped.
4. **Slot reservation** — `RankTrackingRepository.claimDueConfig` updates `next_check_at` and locks the row.
5. **Workflow initiation** — `beginRankCheckRun` attempts `INSERT` into `rank_check_runs`.
6. **Conflict resolution** — On unique‑index violation, stale‑run detection determines retry or skip.

## Key Implementation Files

| File | Responsibility |
|------|--------------|
| [`src/server/features/rank-tracking/services/scheduledRankChecks.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/rank-tracking/services/scheduledRankChecks.ts) | Budget, deadline, plan checks, and scheduler orchestration. |
| [`src/server/features/rank-tracking/services/rankCheckRunGuards.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/rank-tracking/services/rankCheckRunGuards.ts) | Stale‑run detection, workflow status verification, and unique‑index conflict handling. |
| [`src/server/features/rank-tracking/repositories/RankTrackingRepository.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/rank-tracking/repositories/RankTrackingRepository.ts) | Database operations: `claimDueConfig`, `tryCreateRun`, `getActiveRunForConfig`. |
| [`src/db/app.schema.ts`](https://github.com/every-app/open-seo/blob/main/src/db/app.schema.ts) / [`src/db/pg/app.schema.ts`](https://github.com/every-app/open-seo/blob/main/src/db/pg/app.schema.ts) | Partial unique index definition for run exclusivity. |
| [`src/server/features/rank-tracking/services/RankTrackingService.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/rank-tracking/services/RankTrackingService.ts) | Service‑layer consumption of repository guards. |

## Summary

- **Task‑unit budget** caps keyword‑device combinations per tick at 1,000.
- **Wall‑clock deadline** terminates ticks exceeding 3 minutes.
- **Partial unique index** enforces single active run per config at the database level.
- **Stale‑run detection** automatically recovers slots from crashed or hung workflows.
- **Paid‑plan logic** prevents API calls for free accounts or empty configurations.

Together, these mechanisms ensure the rank tracking system respects provider rate limits, minimizes redundant API consumption, and self‑heals from failure scenarios.

## Frequently Asked Questions

### What happens if two scheduler instances try to start the same rank check simultaneously?

The **partial unique index** on `rank_check_runs (config_id) WHERE status IN ('pending','running')` guarantees that only one `INSERT` succeeds. The second receives a constraint violation, which the guard interprets as `"already_running"` and skips. This database‑level enforcement is immune to race conditions between processes.

### How does the system recover if a rank check workflow crashes without updating its status?

`getStaleRankCheckRunReason` in [`rankCheckRunGuards.ts`](https://github.com/every-app/open-seo/blob/main/rankCheckRunGuards.ts) queries the workflow status via `getRankCheckWorkflowStatus`. If the workflow is missing, terminated, or errored, the run is marked failed through `failRunIfActive`, releasing the unique‑index slot. A subsequent scheduler tick can then claim and restart the configuration.

### Why use a partial unique index instead of application‑level locking?

Application locks (Redis, in‑memory, or advisory locks) **fail across process restarts and network partitions**. The PostgreSQL unique index persists with the data, survives crashes, and requires no additional infrastructure. It also provides **exactly‑once semantics** without the complexity of distributed lock coordination.

### Can the budget and deadline constants be tuned for different deployment scenarios?

Yes. `SCHEDULED_TASK_UNIT_BUDGET` and `TICK_DEADLINE_MS` are defined as constants in [`scheduledRankChecks.ts`](https://github.com/every-app/open-seo/blob/main/scheduledRankChecks.ts). Adjusting these values changes the throughput ceiling and maximum tick duration. However, modifications should align with your SERP API provider's rate limits and the worker pool's capacity to avoid throttling or timeouts.