# How OpenSEO Fetches and Processes Backlink Data: DataForSEO Integration Explained

> Discover how OpenSEO fetches and processes backlink data via DataForSEO integration. Learn about its caching, API requests, and response normalization for efficient backlink analysis.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: internals
- Published: 2026-08-09

---

**OpenSEO retrieves backlink data through a layered service architecture that caches results in R2 for 6 hours before making HTTPS requests to the DataForSEO API, then normalizes responses through dedicated mapping functions.**

Backlink analysis is core to modern SEO tooling, and the `every-app/open-seo` repository implements this through a clean separation of concerns across caching, external API communication, and data transformation. This article examines exactly how backlink data flows through the system—from cache key generation to the final paginated result shape.

## The BacklinksService Entry Point

All backlink operations begin in [`src/server/features/backlinks/services/BacklinksService.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/backlinks/services/BacklinksService.ts). This service serves as the public façade, handling cache key construction and dispatching to specialized fetchers.

### Cache Key Construction

Before any external request, the service generates a cache key via `buildCacheKey` that incorporates:

- Organization identifier
- Target URL
- Request parameters
- Spam filter options (when applied)

This ensures filtered and unfiltered views are cached separately, preventing cross-contamination.

### Service Methods

`BacklinksService` exposes four primary operations that delegate to [`backlinksServiceData.ts`](https://github.com/every-app/open-seo/blob/main/backlinksServiceData.ts):

| Method | Purpose |
|--------|---------|
| `profileBacklinksOverview` | Domain-level summary with trends |
| `profileBacklinksRowsPage` | Paginated individual backlink rows |
| `profileReferringDomainsPage` | Aggregated per-domain metrics |
| `profileTopPagesPage` | Top pages by backlink count |

## Fetching Data from DataForSEO

The concrete implementation lives in [`src/server/features/backlinks/services/backlinksServiceData.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/backlinks/services/backlinksServiceData.ts). Each function instantiates a **DataForSEO client** through `createDataforseoClient(billingCustomer)`, passing the organization's billing context for credit tracking.

### Available Endpoints

The client hits five distinct DataForSEO endpoints based on operation type:

1. **`backlinks.summary`** — Returns total backlinks, referring domains, domain rank, and related metrics
2. **`backlinks.history`** — Time-series statistics (used exclusively for domain-scope targets to generate trend data)
3. **`backlinks.rows`** — Paginated list of individual backlinks with full metadata
4. **`backlinks.referringDomains`** — Domain-aggregated backlink counts and authority scores
5. **`backlinks.domainPages`** — Internal pages ranked by inbound link volume

All requests route over HTTPS and are automatically billed against the caller's credit feature (e.g., `"backlinks"`).

## Caching Strategy with R2

Response caching operates through [`src/server/lib/r2-cache.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/r2-cache.ts), invoked via `getCached` and `setCached` helpers.

### TTL Configuration

Two constants govern cache longevity:

- `BACKLINKS_OVERVIEW_TTL_SECONDS` — **6 hours** for overview/summary data
- `BACKLINKS_TAB_TTL_SECONDS` — **6 hours** for paginated tabular data

Identical requests hit cache first, bypassing DataForSEO entirely when fresh data exists. This reduces API costs and improves response latency.

## Data Transformation Pipeline

Raw API payloads undergo systematic normalization before reaching consumers. The mapping layer handles field renaming, null-safety, and structural reshaping.

### Key Mapping Functions

- **`mapBacklinksRows`** — Transforms individual backlink items into clean objects with `domain`, `url`, `anchor`, `rank`, `spamScore`, and timestamp fields
- **`mapReferringDomainsRows`** — Aggregates metrics per referring domain
- **`mapTopPagesRows`** — Extracts page-level backlink counts and authority data
- **`buildOverviewResult`** — Assembles high-level overview combining summary snapshot with trend series from history data

Field names convert from snake_case API responses to camelCase (e.g., `domain_from` → `domainFrom`), and undefined values become explicit `null` for type safety.

## Spam Filtering Implementation

Optional filtering integrates through `BacklinksSpamFilterOptions`, normalized by `normalizeBacklinksSpamFilterOptions`. Available options include:

- `hideSpam: boolean` — Exclude backlinks exceeding spam threshold
- `spamThreshold: number` — Custom cutoff (default: 30)

These parameters join the cache key composition, ensuring filtered results don't pollute unfiltered caches.

## Unified Response Shape

All pageable endpoints return a consistent pagination envelope:

```typescript
{
  rows: TRow[];
  totalCount: number | null;
  hasMore: boolean;
  page: number;
  pageSize: number;
  fetchedAt: string; // ISO 8601 timestamp
}

```

This standardization enables the frontend to render tables, infinite scroll, and CSV exports without conditional logic per endpoint.

## Code Examples

### Domain-Level Overview

```typescript
await BacklinksService.profileOverview(
  { target: "example.com", scope: "domain" },
  billingCustomer,
  "backlinks", // credit feature for billing
);

```

### Paginated Backlink Rows with Spam Filtering

```typescript
await BacklinksService.profileBacklinksPage(
  {
    target: "example.com",
    scope: "domain",
    page: 1,
    pageSize: 50,
    sortField: "firstSeen",
    sortOrder: "desc",
    filters: {},
    mode: "one_per_domain", // or "as_is" for all links
  },
  billingCustomer,
  { hideSpam: true, spamThreshold: 30 },
);

```

### Referring Domain Aggregates

```typescript
await BacklinksService.profileReferringDomainsPage(
  {
    target: "example.com",
    scope: "domain",
    page: 1,
    pageSize: 25,
    sortField: "backlinks",
    sortOrder: "desc",
    filters: {},
  },
  billingCustomer,
);

```

## Core Files Reference

| Component | File Path |
|-----------|-----------|
| Service façade | [`src/server/features/backlinks/services/BacklinksService.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/backlinks/services/BacklinksService.ts) |
| Fetching & caching logic | [`src/server/features/backlinks/services/backlinksServiceData.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/backlinks/services/backlinksServiceData.ts) |
| DataForSEO client factory | [`src/server/lib/dataforseo.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo.ts) |
| R2 cache utilities | [`src/server/lib/r2-cache.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/r2-cache.ts) |
| Type schemas | [`src/types/schemas/backlinks.ts`](https://github.com/every-app/open-seo/blob/main/src/types/schemas/backlinks.ts) |

## Summary

- **OpenSEO backlink data** flows through a three-tier architecture: service layer → cache layer → external API
- **Cache keys** incorporate organization, target URL, parameters, and spam filters for precise invalidation
- **DataForSEO** provides five dedicated endpoints consumed via `createDataforseoClient`
- **6-hour R2 TTL** balances freshness with API cost efficiency
- **Mapping functions** normalize snake_case API responses to camelCase with null-safety
- **Unified pagination shape** enables consistent frontend rendering across all backlink views

## Frequently Asked Questions

### What external API does OpenSEO use for backlink data?

OpenSEO integrates with **DataForSEO**, a specialized SEO data provider. The connection is established through `createDataforseoClient` in [`src/server/lib/dataforseo.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo.ts), which handles authentication, billing context, and HTTPS communication to endpoints like `backlinks.summary`, `backlinks.rows`, and `backlinks.referringDomains`.

### How does OpenSEO handle backlink API rate limits?

Rate limiting is managed through **R2 caching with 6-hour TTL**. Before any external request, `BacklinksService` checks `getCached` using a deterministic cache key. Cache hits return immediately without consuming DataForSEO credits. This strategy significantly reduces actual API calls while maintaining reasonably fresh data.

### Can spam backlinks be filtered in OpenSEO requests?

Yes. The system accepts `BacklinksSpamFilterOptions` including `hideSpam` boolean and `spamThreshold` number (default 30). These options pass through `normalizeBacklinksSpamFilterOptions` and are incorporated into cache keys, ensuring filtered and unfiltered datasets remain isolated in cache.

### What caching infrastructure powers OpenSEO's backlink service?

OpenSEO uses **Cloudflare R2** for object storage, abstracted through [`src/server/lib/r2-cache.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/r2-cache.ts). The cache layer provides `getCached`, `setCached`, and `buildCacheKey` utilities that the backlink service invokes before and after DataForSEO requests.