# How External Services Are Integrated into Open‑SEO: Architecture and Implementation

> Learn how Open-SEO integrates external services using a typed wrapper layer for authentication, validation, retries, and uniform async client factories. Discover the architecture and implementation details.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: architecture
- Published: 2026-06-28

---

**Open‑SEO integrates with third-party APIs through a thin, typed wrapper layer that centralizes authentication, validates responses with Zod schemas, retries transient errors, and exposes uniform async client factories to the rest of the application.**

The every-app/open-seo repository demonstrates exactly how external services are integrated into open-seo using a consistent architectural pattern. Instead of scattering raw `fetch` calls throughout the codebase, the project isolates third-party interactions in dedicated library files under `src/server/lib/`. This design ensures that authentication, validation, and error classification happen transparently before data ever reaches the core business logic.

## Unified Integration Architecture

All external service integrations in Open‑SEO share six common design goals implemented through reusable patterns.

### Typed Request and Response Validation

Every SDK client is wrapped with **Zod** schemas that validate raw JSON before it reaches the core app. For example, DataForSEO responses are validated against `serpSnapshotItemSchema` in [`src/server/lib/dataforseo/serp.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/serp.ts) before being returned to callers.

### Centralized Authentication

Each provider uses a single helper to inject credentials. DataForSEO relies on `getRequiredEnvValue("DATAFORSEO_API_KEY")` from [`src/server/lib/dataforseo/core.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/core.ts), while Google Search Console uses `getAuth().api.getAccessToken` from [`src/server/lib/gscClient.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/gscClient.ts) to mint OAuth tokens dynamically.

### Error Handling and Classification

A generic `createAuthenticatedFetch` utility catches non‑2xx responses, retries transient 5xx errors with exponential back-off, and maps known status codes to product‑specific `AppError` objects. This prevents leaked implementation details from bubbling up to the user interface.

### Strategic Caching

KV‑based caching eliminates redundant network calls. The Ahrefs integration in [`src/serverFunctions/ahrefs.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/ahrefs.ts) stores Domain Rating lookups with a 24‑hour TTL, checking `env.KV.get(cacheKey)` before issuing any external request.

### Rate‑Limit Safety

Long‑running calls use `AbortSignal.timeout` (e.g., DataForSEO’s 60‑second timeout) and back‑off logic to prevent hanging requests from degrading application performance.

### Uniform Client API

Each service exposes a small `create…Client` factory that returns plain async functions such as `listSites`, `querySearchAnalytics`, or `inspectUrl`, creating a predictable interface for the rest of the application.

## DataForSEO Integration

DataForSEO powers keyword SERP analysis, rank checking, backlink data, and AI‑search features through a robust wrapper layer.

### Core Fetch Layer with Automatic Retries

All DataForSEO SDK calls flow through the custom fetch implementation in **src/server/lib/dataforseo/core.ts**. This utility injects the Base64‑encoded API key into the `Authorization` header, retries on 5xx status codes, and throws structured `AppError` instances for other failures.

```typescript
// src/server/lib/dataforseo/core.ts
const apiKey = await getRequiredEnvValue("DATAFORSEO_API_KEY");
headers.set("Authorization", `Basic ${apiKey}`);
const response = await fetch(url, { … });

```

### Account State Verification

Before any paid endpoint is invoked, the helper `fetchDataforseoAccountState` in **src/server/lib/dataforseoAccountState.ts** verifies the user’s subscription status by querying the DataForSEO appendix endpoint.

```typescript
const response = await fetch(`${API_BASE}/v3/appendix/user_data`, { … });

```

### Live SERP and Rank Check Operations

The `fetchLiveSerp` function in **src/server/lib/dataforseo/serp.ts** constructs live organic SERP requests, validates each result item against `serpSnapshotItemSchema`, and returns a fully typed payload. For bulk operations, `postRankCheckTasks` builds batches of `googleOrganicTaskPost` requests, encodes correlation tags, and returns billing information in a single network round‑trip.

## Google Search Console (GSC) Integration

GSC data is fetched via the Google Webmasters REST API using a dedicated client that handles OAuth token rotation transparently.

### OAuth Token Management

The `createGscClient` factory in **src/server/lib/gscClient.ts** mints fresh access tokens using Better Auth before every request.

```typescript
const token = await getAuth().api.getAccessToken({
  body: { providerId: GSC_OAUTH_PROVIDER_ID, userId: opts.userId },
});

```

### Authenticated Request Wrapper

An internal `request` helper attaches the `Bearer` token, sets `Content-Type` headers for POST bodies, and throws `GscApiError` instances with human‑friendly messages determined by `messageForStatus`.

```typescript
const response = await fetch(url, {
  method: init?.method ?? "GET",
  headers: { Authorization: `Bearer ${token}`, … },
  body: hasBody ? JSON.stringify(init?.body) : undefined,
});

```

### Public API Methods

The client exposes three primary methods used throughout Open‑SEO:
- `listSites()` – returns verified site URLs and permission levels
- `querySearchAnalytics(params)` – fetches click, impression, and position data
- `inspectUrl(url)` – retrieves indexing status and enhancements

## Ahrefs Domain Rating Integration

Ahrefs is used solely to enrich the Backlinks table with free Domain Rating (DR) scores, implemented with aggressive caching to minimize API usage.

### Cache‑First Lookup Strategy

The `resolveDomainRating` function in **src/serverFunctions/ahrefs.ts** first checks a Cloudflare KV cache using `env.KV.get(cacheKey)`. Cached values are stored as JSON strings with a 24‑hour TTL. Only cache misses trigger network requests.

```typescript
const cached = await env.KV.get(cacheKey);
if (cached !== null) return parseCachedRating(cached);

```

### Batch Processing and Normalization

The `fetchDomainRating` helper calls the Ahrefs public endpoint with a 5‑second timeout, validates the JSON against `ahrefsResponseSchema`, and converts a rating of `0` to `null` (indicating no rating available). For efficiency, `getAhrefsDomainRatings` normalizes input domains and batches them into groups of 20, ensuring a single network call per distinct domain while merging results back onto the original list.

## Additional Third‑Party Services

Beyond the major SEO providers, Open‑SEO integrates several lightweight services following the same wrapper pattern:

- **Plausible Analytics** – [`web/src/routes/api/event.ts`](https://github.com/every-app/open-seo/blob/main/web/src/routes/api/event.ts) POSTs to `https://plausible.io/api/event` for privacy‑friendly tracking.
- **Loops Email** – [`web/src/routes/api/subscribe.ts`](https://github.com/every-app/open-seo/blob/main/web/src/routes/api/subscribe.ts) syncs contacts via `https://app.loops.so/api/v1/contacts/create`.
- **Reddit Conversions** – [`src/server/lib/reddit-conversions.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/reddit-conversions.ts) sends pixel events to Reddit’s Ads API.
- **Autumn Billing** – [`src/server/billing/autumn-webhook.ts`](https://github.com/every-app/open-seo/blob/main/src/server/billing/autumn-webhook.ts) receives webhook POSTs from `https://api.useautumn.com`.

Each implementation builds the request, injects environment‑based credentials, and performs simple `fetch` calls with error checking consistent with the core architecture.

## Summary

- **Open‑SEO isolates external service logic** in dedicated files under `src/server/lib/` and `src/serverFunctions/`, preventing API specifics from leaking into business logic.
- **Validation happens at the boundary** using Zod schemas that guarantee type safety before data enters the core application.
- **Authentication is centralized** through helpers like `getRequiredEnvValue` and `getAuth().api.getAccessToken`, ensuring tokens are refreshed and injected consistently.
- **Resilience is built‑in** via automatic retries for 5xx errors, `AbortSignal.timeout` for rate‑limit protection, and KV caching for expensive calls like Ahrefs Domain Rating lookups.

## Frequently Asked Questions

### How does Open‑SEO handle authentication for external APIs?

Authentication is centralized through dedicated helpers. DataForSEO uses `getRequiredEnvValue` to inject API keys from environment variables, while Google Search Console uses `getAuth().api.getAccessToken` to mint OAuth tokens dynamically before each request. This pattern ensures credentials are never hard‑coded and are refreshed automatically.

### What caching strategy does Open‑SEO use for external service calls?

Open‑SEO uses Cloudflare KV caching for expensive or rate‑limited endpoints. The Ahrefs integration demonstrates this pattern: `resolveDomainRating` checks `env.KV.get(cacheKey)` first and only issues a network request on cache miss, storing results with a 24‑hour TTL. This minimizes redundant API calls and improves response times.

### How are errors from third‑party APIs handled and classified?

Errors are caught by a generic `createAuthenticatedFetch` wrapper that intercepts non‑2xx responses, retries transient 5xx errors, and maps known status codes to typed `AppError` objects. For example, DataForSEO calls in [`src/server/lib/dataforseo/core.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/core.ts) throw structured errors that the application can catch and display without exposing raw implementation details.

### Where are the main integration files located in the Open‑SEO codebase?

Core integration logic resides in [`src/server/lib/dataforseo/core.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/core.ts) (DataForSEO fetch layer), [`src/server/lib/gscClient.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/gscClient.ts) (Google Search Console OAuth and methods), and [`src/serverFunctions/ahrefs.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/ahrefs.ts) (Ahrefs Domain Rating with KV caching). Lightweight integrations for Plausible, Loops, Reddit, and Autumn are located in `web/src/routes/api/` and `src/server/billing/` respectively.