# How OpenSEO Performs Backlink Analysis Using DataForSEO: A Technical Deep Dive

> Discover how OpenSEO leverages DataForSEO for advanced backlink analysis. Explore the seven-stage pipeline for efficient and accurate data processing.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: deep-dive
- Published: 2026-09-01

---

**OpenSEO performs backlink analysis using DataForSEO through a seven-stage pipeline that includes input validation, optional Turnstile verification, rate-limiting, edge caching, payload construction, parallel API calls to DataForSEO's summary and backlinks endpoints, and schema-driven result parsing.**

OpenSEO is an open-source SEO platform that leverages the DataForSEO API to deliver comprehensive backlink insights. Understanding how OpenSEO performs backlink analysis using DataForSEO reveals a carefully architected edge-computing pipeline designed for cost efficiency and reliability. The implementation in [`web/src/routes/api/backlink-check.ts`](https://github.com/every-app/open-seo/blob/main/web/src/routes/api/backlink-check.ts) demonstrates production-grade patterns for API consumption, caching, and request validation.

## The Seven-Stage Backlink Analysis Pipeline

### 1. Input Validation and Domain Normalization

Every request begins with **Zod schema validation** using `requestSchema` to ensure the request body contains a valid target domain. The system then calls `normalizeDomain()` to trim whitespace, remove `www.` prefixes, and lowercase the hostname. This normalization guarantees consistent cache keys and API queries regardless of input formatting.

### 2. Human Verification via Turnstile

When the `TURNSTILE_SECRET_KEY` environment variable is configured, the endpoint verifies the client token against Cloudflare's Turnstile API. This step blocks automated abuse without affecting local development workflows where the secret key may be omitted.

### 3. Rate Limiting and Daily Budget Enforcement

OpenSEO implements a dual-layer protection strategy. A generic `RateLimiter` controls per-IP request frequency, while a Cloudflare KV store (`BACKLINK_CHECK_KV`) tracks consumption against `DAILY_CHECK_BUDGET` (set to 500 checks). Once the budget is exhausted, the endpoint returns **HTTP 429** to prevent DataForSEO overages.

### 4. Edge Caching Strategy

The Cloudflare edge cache is consulted first via `cache.match`. Successful responses cache for `CACHE_TTL_SECONDS` (24 hours), while failed requests cache for 2 minutes using negative caching. This prevents retry storms during DataForSEO outages and eliminates redundant API calls for popular domains.

### 5. Research Scope Payload Construction

The `commonPayload` object applies consistent research parameters across the application using the `ResearchScope` type definitions. The payload includes:

- `include_subdomains: true`
- `include_indirect_links: true`
- `exclude_internal_backlinks: true`
- `backlinks_status_type: "live"`
- `rank_scale: "one_hundred"`

These settings ensure that backlink queries follow the same location and language rules used for keyword research elsewhere in the platform.

### 6. Parallel DataForSEO API Calls

Two POST requests execute simultaneously using `Promise.all`:

- `/v3/backlinks/summary/live` returns aggregate metrics including domain rank, total backlink count, referring domains, and broken backlinks.
- `/v3/backlinks/backlinks/live` fetches the top 15 backlink items using `mode: "one_per_domain"` and ordered by `domain_from_rank,desc`.

Both calls use **Basic Authentication** (`Authorization: Basic ${apiKey}`) built from the `DATAFORSEO_API_KEY` environment variable. Responses validate against `taskEnvelopeSchema`, and non-20000 status codes raise immediate errors.

### 7. Schema-Driven Result Parsing

Zod schemas (`summaryResultSchema` and `backlinksResultSchema`) safely extract fields from the API responses. The endpoint constructs a concise JSON payload containing:

- Domain summary statistics
- An array of top backlinks with `domainFrom`, `urlFrom`, `urlTo`, `pageTitle`, `anchor`, `dofollow` flags, and `domainRank`

The response is cached with `Cache-Control: public, max-age=86400` before returning to the client.

## Cost Control and Architectural Design

**Daily Budget Protection:** The KV-backed counter enforces a hard limit of 500 daily checks. At approximately $0.04 per call, this caps DataForSEO spending at roughly $20 per day while allowing sufficient capacity for production use.

**Stateless Edge Architecture:** All state management—including caching, rate limiting, and budget tracking—occurs at the Cloudflare edge. This design eliminates backend state, minimizes latency, and allows the system to scale horizontally without database dependencies.

**Schema Safety:** Every external payload undergoes Zod validation, making the system resilient against DataForSEO API changes and ensuring that only well-formed data propagates through the pipeline.

## Implementation Examples

### Client-Side Integration

```tsx
import { useState } from "react";

export function BacklinkChecker() {
  const [domain, setDomain] = useState("");
  const [result, setResult] = useState<any>(null);
  const [error, setError] = useState<string>("");

  async function checkBacklinks() {
    try {
      const resp = await fetch("/api/backlink-check", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ target: domain }),
      });
      const data = await resp.json();
      if (!resp.ok) throw new Error(data.error);
      setResult(data);
    } catch (e: any) {
      setError(e.message);
    }
  }

  return (
    <div>
      <input
        value={domain}
        onChange={(e) => setDomain(e.target.value)}
        placeholder="example.com"
      />
      <button onClick={checkBacklinks}>Check Backlinks</button>
      {error && <p className="error">{error}</p>}
      {result && (
        <pre>{JSON.stringify(result, null, 2)}</pre>
      )}
    </div>
  );
}

```

The React component posts a JSON payload containing the `target` domain. The server returns data matching the shape defined in `summaryResultSchema` and `backlinksResultSchema`.

### Server-Side Core Logic

```ts
// Build the shared payload for both summary & backlink list
const commonPayload = {
  target: domain,
  include_subdomains: true,
  include_indirect_links: true,
  exclude_internal_backlinks: true,
  backlinks_status_type: "live",
  rank_scale: "one_hundred",
};

// Parallel fetch of summary and top-backlink list
const [summaryRaw, backlinksRaw] = await Promise.all([
  fetchDataforseoResult("/v3/backlinks/summary/live", commonPayload, apiKey),
  fetchDataforseoResult(
    "/v3/backlinks/backlinks/live",
    {
      ...commonPayload,
      limit: TOP_BACKLINKS_LIMIT,
      mode: "one_per_domain",
      order_by: ["domain_from_rank,desc"],
    },
    apiKey,
  ),
]);

// Validate and shape the data
const summary   = summaryResultSchema.parse(summaryRaw ?? {});
const backlinks = backlinksResultSchema.parse(backlinksRaw ?? {});

const topBacklinks = (backlinks.items ?? [])
  .filter(i => i.type === "backlink" && i.url_from)
  .map(i => ({
    domainFrom: i.domain_from ?? null,
    urlFrom:    i.url_from ?? null,
    urlTo:      i.url_to ?? null,
    pageTitle:  i.page_from_title?.trim() ? i.page_from_title : null,
    anchor:     i.anchor?.trim() ? i.anchor : null,
    dofollow:   i.dofollow ?? null,
    domainRank: i.domain_from_rank ?? null,
  }));

```

This excerpt from [`web/src/routes/api/backlink-check.ts`](https://github.com/every-app/open-seo/blob/main/web/src/routes/api/backlink-check.ts) demonstrates the payload creation, parallel DataForSEO calls, Zod parsing, and final transformation that powers the JSON response.

## Summary

- OpenSEO validates and normalizes domains using Zod schemas before processing any DataForSEO requests
- Cloudflare Turnstile integration prevents abuse without impacting local development workflows
- A dual-layer rate limiter combining per-IP restrictions and a daily KV-backed budget controls DataForSEO costs at approximately $0.04 per call
- 24-hour edge caching with 2-minute negative caching reduces redundant API calls for identical domains
- Parallel requests to `/v3/backlinks/summary/live` and `/v3/backlinks/backlinks/live` optimize latency through `Promise.all`
- Schema-driven parsing using `summaryResultSchema` and `backlinksResultSchema` ensures type safety across the entire data pipeline

## Frequently Asked Questions

### What DataForSEO endpoints does OpenSEO use for backlink analysis?

OpenSEO uses two endpoints: `/v3/backlinks/summary/live` for aggregate metrics including domain rank and referring domain counts, and `/v3/backlinks/backlinks/live` for detailed backlink items. Both endpoints are called in parallel to minimize response latency, with the latter configured to return one result per domain ordered by domain rank.

### How does OpenSEO prevent excessive DataForSEO API costs?

The system implements a daily budget counter using Cloudflare KV store (`BACKLINK_CHECK_KV`) that limits checks to 500 per day. Combined with 24-hour edge caching and per-IP rate limiting, this ensures costs remain predictable and prevents runaway API usage that could exhaust the underlying DataForSEO account balance.

### Can I use OpenSEO's backlink checker without Cloudflare Turnstile?

Yes. The Turnstile verification step is optional and only activates when the `TURNSTILE_SECRET_KEY` environment variable is configured. Local development works without this key, while production deployments can enable it to prevent automated abuse without modifying application code.

### What data does OpenSEO return from the backlink analysis?

The API returns a JSON object containing the target domain, summary statistics (total backlinks, referring domains, broken links, domain rank), and an array of top backlinks. Each backlink includes the source domain and URL, destination URL, page title, anchor text, dofollow status, and domain rank score as defined in the `backlinksResultSchema` type definitions.