OpenSEO DataForSEO API Integration: Architecture and Abstraction Layers

OpenSEO wraps the DataForSEO API in a multi-layered TypeScript stack that adds typed envelopes, R2 caching, billing classification, and domain-specific modules to transform raw API calls into safe, metered, and cacheable internal services.

OpenSEO (from the every-app/open-seo repository) integrates with the DataForSEO platform through a thin, well-structured wrapper located under src/server/lib/dataforseo. Rather than calling the raw SDK directly, the codebase implements a sophisticated integration pattern that emphasizes type safety, error resilience, and cost control through strategic abstraction layers.

Architectural Overview of the Integration

The integration architecture follows a layered approach where each tier adds specific functionality to the raw DataForSEO API. This design ensures consistent TypeScript typings via Zod, centralizes error handling through the DataforseoErrorClassifier, and implements a cache-first strategy to minimize redundant API calls.

Client Factory and Authentication

The foundation of the integration resides in src/server/lib/dataforseo/client.ts, which instantiates the low-level client with project-specific API credentials. This factory function injects request-level timeouts and returns a configured wrapper used throughout the application stack.

The client creation abstracts the authentication details away from business logic, allowing components to simply call createDataforseoClient() with environment-derived credentials while the factory handles the underlying dataforseo-client initialization.

Core API Helpers

Located in src/server/lib/dataforseo/core.ts, this layer holds the raw HTTP-style helpers including serpApi, labsApi, onPageApi, and keywordsDataApi. All endpoint URLs are constructed here following the pattern https://api.dataforseo.com/v3/{service}/{endpoint}.

These core functions serve as the HTTP transport layer, providing the basic connectivity that higher-level modules consume while remaining agnostic to business logic concerns like caching or billing.

The Typed Envelope Layer

The src/server/lib/dataforseo/envelope.ts file implements the critical envelope pattern that wraps every request and response. This layer validates payloads with Zod schemas, classifies errors via DataforseoErrorClassifier, and surfaces a uniform DataforseoResult<T> type across the entire application.

When API failures occur, the envelope maps HTTP errors to known DataforseoErrorCode values, enabling consistent error handling downstream. It also applies retry logic for transient failures before surfacing permanent errors to calling code.

Additional Abstraction Layers

Beyond the basic client and envelope, OpenSEO adds three significant infrastructure layers that differentiate it from a thin API wrapper.

R2 Caching Strategy

The system implements an R2-backed key-value cache defined in src/server/lib/r2-cache.ts, utilizing the prefix dataforseo-cache/ for all stored results. Before executing any network call, domain modules check this cache using keys derived from request parameters.

This cache-first approach reduces API costs and improves response times for repeated queries. The caching layer supports TTL configuration, allowing different DataForSEO endpoints to specify appropriate freshness windows based on data volatility.

Billing Classification System

Usage metering operates through src/server/lib/dataforseoBillingClassification.ts, which maps API response paths like ["v3","dataforseo_labs","google","related_keywords","live"] to internal product features. This mapping enables per-feature metering that links directly to user subscription plans.

When the envelope layer successfully receives a response, the billing classifier inspects the endpoint path and increments the appropriate usage counter, ensuring accurate cost attribution for multi-tenant billing scenarios.

Domain-Specific Module Facades

High-level functionality is organized into focused modules that present clean APIs to the rest of the application:

• src/server/lib/dataforseo/serp.ts: Exports fetchSerpResults for search engine result page data
• src/server/lib/dataforseo/labs.ts: Provides fetchKeywordOverview and keyword research functions
• src/server/lib/dataforseo/google-ads.ts: Handles Google Ads data retrieval
• src/server/lib/dataforseo/backlinks.ts: Manages backlink analysis operations
• src/server/lib/dataforseo/lighthouse.ts: Integrates Lighthouse auditing via DataForSEO

Each module normalizes the raw API responses into internal TypeScript types defined in src/types/schemas/*.ts, ensuring that UI components receive consistently shaped data regardless of external API variations.

Implementation Walkthrough

The following examples demonstrate how these layers work together in practice, from client initialization to cached data retrieval.

Creating a DataForSEO Client

import { createDataforseoClient } from '@/server/lib/dataforseo';

// Factory reads credentials from environment variables
const dfseo = createDataforseoClient({
  apiKey: process.env.DATAFORSEO_API_KEY!,
  apiSecret: process.env.DATAFORSEO_API_SECRET!,
});

This initialization creates a client instance that inherits timeout configurations and authentication headers required for all subsequent API interactions.

Fetching SERP Results with Full Stack Integration

import { fetchSerpResults } from '@/server/lib/dataforseo/serp';

const results = await fetchSerpResults(dfseo, {
  target: 'https://example.com',
  keywords: ['open seo', 'dataforseo integration'],
  locationCode: 2840,  // United States
  languageCode: 1000,  // English
});

Behind this single call, the execution flow traverses all abstraction layers: the SERP module prepares the payload, the envelope validates and wraps the request, the cache layer checks for existing results under dataforseo-cache/serp/..., and upon successful API response, the billing classifier updates usage metrics for the SERP feature.

Retrieving Keyword Lab Data

import { fetchKeywordOverview } from '@/server/lib/dataforseo/labs';

const overview = await fetchKeywordOverview(dfseo, {
  keywords: ['open seo'],
  country: 'us',
});

This function calls labsApi targeting /v3/dataforseo_labs/google/keyword_overview/live, validates the response against KeywordOverviewResponseSchema, and returns a typed array of KeywordOverviewItem objects ready for UI consumption.

Implementing Cache-First Strategies

import { r2Cache } from '@/server/lib/r2-cache';
import { fetchSerpResults } from '@/server/lib/dataforseo/serp';

async function getSerpWithCache(cacheKey: string, params: any) {
  // Check R2 cache first
  const cached = await r2Cache.get(`dataforseo-cache/serp/${cacheKey}`);
  if (cached) return JSON.parse(cached);

  // Fetch fresh data through the envelope layer
  const fresh = await fetchSerpResults(dfseo, params);
  
  // Store with 1-hour TTL
  await r2Cache.put(
    `dataforseo-cache/serp/${cacheKey}`, 
    JSON.stringify(fresh), 
    { ttl: 3600 }
  );
  
  return fresh;
}

This pattern exemplifies how OpenSEO layers caching logic on top of the envelope system, ensuring expensive DataForSEO API calls are only made when necessary while maintaining data freshness controls.

Summary

• Client Factory: Centralized authentication and timeout configuration in src/server/lib/dataforseo/client.ts
• Typed Envelope: Zod-validated request/response wrappers with standardized error classification via DataforseoErrorClassifier
• R2 Caching: Persistent cache layer using dataforseo-cache/ prefix to minimize API costs and latency
• Billing Integration: Automatic usage metering that maps API paths to billable features through src/server/lib/dataforseoBillingClassification.ts
• Domain Modules: High-level, type-safe functions in feature-specific files (serp.ts, labs.ts, etc.) that expose clean APIs to business logic

Frequently Asked Questions

How does OpenSEO handle DataForSEO API authentication?

OpenSEO centralizes authentication in the createDataforseoClient function within src/server/lib/dataforseo/client.ts. This factory reads the DATAFORSEO_API_KEY and DATAFORSEO_API_SECRET from environment variables, configures the underlying SDK with these credentials, and injects request timeouts. All subsequent API interactions use this pre-configured client instance, ensuring credentials never leak into business logic code.

What error handling mechanisms protect against DataForSEO API failures?

The src/server/lib/dataforseo/envelope.ts file implements a comprehensive error handling strategy that intercepts all HTTP responses before they reach business logic. It classifies errors using DataforseoErrorClassifier to map HTTP status codes and API error messages to standardized DataforseoErrorCode values. The envelope also implements retry logic for transient network failures, only surfacing permanent errors after exhaustion of retry attempts, providing calling code with predictable error types for handling.

Why does OpenSEO use an R2 cache for DataForSEO data?

The R2 caching layer in src/server/lib/r2-cache.ts reduces operational costs and improves response times by storing expensive API results under the dataforseo-cache/ prefix. Since DataForSEO charges per API call and many SEO metrics (like historical rankings or backlink data) remain valid for hours or days, the cache-first strategy allows OpenSEO to serve repeated requests instantly without incurring additional API fees, while TTL controls ensure data freshness for time-sensitive metrics.

How does OpenSEO track usage for billing purposes when calling DataForSEO?

OpenSEO implements usage tracking through src/server/lib/dataforseoBillingClassification.ts, which inspects the API endpoint path after each successful call to determine which internal feature was consumed. For example, a call to ["v3","dataforseo_labs","google","related_keywords","live"] maps to a specific product feature, allowing the system to increment usage counters tied to subscription plans. This occurs automatically within the envelope layer, ensuring every API interaction is properly metered without requiring manual instrumentation in business logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}