How OpenSEO Fetches and Processes Backlink Data: DataForSEO Integration Explained

OpenSEO retrieves backlink data through a layered service architecture that caches results in R2 for 6 hours before making HTTPS requests to the DataForSEO API, then normalizes responses through dedicated mapping functions.

Backlink analysis is core to modern SEO tooling, and the every-app/open-seo repository implements this through a clean separation of concerns across caching, external API communication, and data transformation. This article examines exactly how backlink data flows through the system—from cache key generation to the final paginated result shape.

The BacklinksService Entry Point

All backlink operations begin in src/server/features/backlinks/services/BacklinksService.ts. This service serves as the public façade, handling cache key construction and dispatching to specialized fetchers.

Cache Key Construction

Before any external request, the service generates a cache key via buildCacheKey that incorporates:

• Organization identifier
• Target URL
• Request parameters
• Spam filter options (when applied)

This ensures filtered and unfiltered views are cached separately, preventing cross-contamination.

Service Methods

BacklinksService exposes four primary operations that delegate to backlinksServiceData.ts:

Method	Purpose
profileBacklinksOverview	Domain-level summary with trends
profileBacklinksRowsPage	Paginated individual backlink rows
profileReferringDomainsPage	Aggregated per-domain metrics
profileTopPagesPage	Top pages by backlink count

Fetching Data from DataForSEO

The concrete implementation lives in src/server/features/backlinks/services/backlinksServiceData.ts. Each function instantiates a DataForSEO client through createDataforseoClient(billingCustomer), passing the organization's billing context for credit tracking.

Available Endpoints

The client hits five distinct DataForSEO endpoints based on operation type:

1. backlinks.summary — Returns total backlinks, referring domains, domain rank, and related metrics
2. backlinks.history — Time-series statistics (used exclusively for domain-scope targets to generate trend data)
3. backlinks.rows — Paginated list of individual backlinks with full metadata
4. backlinks.referringDomains — Domain-aggregated backlink counts and authority scores
5. backlinks.domainPages — Internal pages ranked by inbound link volume

All requests route over HTTPS and are automatically billed against the caller's credit feature (e.g., "backlinks").

Caching Strategy with R2

Response caching operates through src/server/lib/r2-cache.ts, invoked via getCached and setCached helpers.

TTL Configuration

Two constants govern cache longevity:

• BACKLINKS_OVERVIEW_TTL_SECONDS — 6 hours for overview/summary data
• BACKLINKS_TAB_TTL_SECONDS — 6 hours for paginated tabular data

Identical requests hit cache first, bypassing DataForSEO entirely when fresh data exists. This reduces API costs and improves response latency.

Data Transformation Pipeline

Raw API payloads undergo systematic normalization before reaching consumers. The mapping layer handles field renaming, null-safety, and structural reshaping.

Key Mapping Functions

• mapBacklinksRows — Transforms individual backlink items into clean objects with domain, url, anchor, rank, spamScore, and timestamp fields
• mapReferringDomainsRows — Aggregates metrics per referring domain
• mapTopPagesRows — Extracts page-level backlink counts and authority data
• buildOverviewResult — Assembles high-level overview combining summary snapshot with trend series from history data

Field names convert from snake_case API responses to camelCase (e.g., domain_from → domainFrom), and undefined values become explicit null for type safety.

Spam Filtering Implementation

Optional filtering integrates through BacklinksSpamFilterOptions, normalized by normalizeBacklinksSpamFilterOptions. Available options include:

• hideSpam: boolean — Exclude backlinks exceeding spam threshold
• spamThreshold: number — Custom cutoff (default: 30)

These parameters join the cache key composition, ensuring filtered results don't pollute unfiltered caches.

Unified Response Shape

All pageable endpoints return a consistent pagination envelope:

{
  rows: TRow[];
  totalCount: number | null;
  hasMore: boolean;
  page: number;
  pageSize: number;
  fetchedAt: string; // ISO 8601 timestamp
}

This standardization enables the frontend to render tables, infinite scroll, and CSV exports without conditional logic per endpoint.

Code Examples

Domain-Level Overview

await BacklinksService.profileOverview(
  { target: "example.com", scope: "domain" },
  billingCustomer,
  "backlinks", // credit feature for billing
);

Paginated Backlink Rows with Spam Filtering

await BacklinksService.profileBacklinksPage(
  {
    target: "example.com",
    scope: "domain",
    page: 1,
    pageSize: 50,
    sortField: "firstSeen",
    sortOrder: "desc",
    filters: {},
    mode: "one_per_domain", // or "as_is" for all links
  },
  billingCustomer,
  { hideSpam: true, spamThreshold: 30 },
);

Referring Domain Aggregates

await BacklinksService.profileReferringDomainsPage(
  {
    target: "example.com",
    scope: "domain",
    page: 1,
    pageSize: 25,
    sortField: "backlinks",
    sortOrder: "desc",
    filters: {},
  },
  billingCustomer,
);

Core Files Reference

Component	File Path
Service façade	src/server/features/backlinks/services/BacklinksService.ts
Fetching & caching logic	src/server/features/backlinks/services/backlinksServiceData.ts
DataForSEO client factory	src/server/lib/dataforseo.ts
R2 cache utilities	src/server/lib/r2-cache.ts
Type schemas	src/types/schemas/backlinks.ts

Summary

• OpenSEO backlink data flows through a three-tier architecture: service layer → cache layer → external API
• Cache keys incorporate organization, target URL, parameters, and spam filters for precise invalidation
• DataForSEO provides five dedicated endpoints consumed via createDataforseoClient
• 6-hour R2 TTL balances freshness with API cost efficiency
• Mapping functions normalize snake_case API responses to camelCase with null-safety
• Unified pagination shape enables consistent frontend rendering across all backlink views

Frequently Asked Questions

What external API does OpenSEO use for backlink data?

OpenSEO integrates with DataForSEO, a specialized SEO data provider. The connection is established through createDataforseoClient in src/server/lib/dataforseo.ts, which handles authentication, billing context, and HTTPS communication to endpoints like backlinks.summary, backlinks.rows, and backlinks.referringDomains.

How does OpenSEO handle backlink API rate limits?

Rate limiting is managed through R2 caching with 6-hour TTL. Before any external request, BacklinksService checks getCached using a deterministic cache key. Cache hits return immediately without consuming DataForSEO credits. This strategy significantly reduces actual API calls while maintaining reasonably fresh data.

Can spam backlinks be filtered in OpenSEO requests?

Yes. The system accepts BacklinksSpamFilterOptions including hideSpam boolean and spamThreshold number (default 30). These options pass through normalizeBacklinksSpamFilterOptions and are incorporated into cache keys, ensuring filtered and unfiltered datasets remain isolated in cache.

What caching infrastructure powers OpenSEO's backlink service?

OpenSEO uses Cloudflare R2 for object storage, abstracted through src/server/lib/r2-cache.ts. The cache layer provides getCached, setCached, and buildCacheKey utilities that the backlink service invokes before and after DataForSEO requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}