DataForSEO Client Integration Architecture in OpenSEO

OpenSEO isolates the 3 MB DataForSEO SDK behind a lazy-loaded, metered client factory that tracks usage credits in hosted mode while remaining transparent in self-hosted deployments.

The every-app/open-seo repository implements a layered integration pattern that decouples the heavy DataForSEO SDK from the core server runtime. This architecture prevents the ~3 MB SDK from inflating cold-start times while ensuring every API call is properly metered against organizational credit pools in managed deployments.

The Metered Client Factory

At the center of the integration sits createDataforseoClient, defined in src/server/lib/dataforseo/client.ts. This factory function accepts a billing customer context and returns a namespaced client exposing domains like business, backlinks, keywords, domain, serp, labs, lighthouse, and aiSearch.

Every method on these namespaces is wrapped by the meter function. This wrapper determines the runtime mode via isHostedServerAuthMode from src/server/lib/runtime-env.ts. In hosted mode, it records costs against credit features such as rank_tracking or local_seo through trackDataforseoCost; in self-hosted mode, it forwards calls directly to DataForSEO without metering.

Lazy Loading and Section Isolation

To prevent the DataForSEO SDK from bloating the server bundle, OpenSEO employs a dynamic import strategy. The loadDataforseoSections function triggers a lazy import() of src/server/lib/dataforseo/sections.ts only upon first use.

The sections.ts file acts as a barrel export for concrete fetch functions like fetchLiveSerp, fetchBacklinksRows, and fetchRankCheckTaskPost. This ensures the ~3 MB SDK lives in its own isolated chunk, loaded once and cached for subsequent calls.

Request Pipeline and Error Standardization

The architecture enforces a strict three-tier data flow:

• Section Fetchers: Files like src/server/lib/dataforseo/serp.ts, backlinks.ts, and labs.ts contain thin wrappers that validate inputs and convert SDK responses into OpenSEO schemas.
• Core HTTP Layer: src/server/lib/dataforseo/core.ts handles raw network I/O, building request URLs, injecting authentication headers, and attaching request IDs for tracing.
• Envelope Pattern: src/server/lib/dataforseo/envelope.ts defines DataforseoApiResponse<T> and DataforseoChargedTaskError, providing the assertOk helper to standardize success and error handling across all sections.

This design guarantees consistent error shapes and enables billing logic to inspect error.billing for charged-task scenarios.

Runtime Mode Detection and Billing

The integration adapts its behavior based on deployment context. src/server/lib/runtime-env.ts exports isHostedServerAuthMode, which the meterDataforseoCall function checks before executing requests.

When running in hosted mode, the system maps DataForSEO API paths to internal credit features defined in src/server/billing/subscription.ts. After a successful call or a caught DataforseoChargedTaskError, trackDataforseoCost records the USD cost against the organization’s usage-credit pool.

End-to-End Data Flow

A typical request traverses the following path:

1. A server tool invokes createDataforseoClient(customer) to obtain a metered client.
2. The tool calls a namespaced method like client.serp.rankCheckTaskPost({ ... }).
3. The meter wrapper detects hosted mode, lazily loads sections.ts if needed, and delegates to sections.fetchRankCheckTaskPost.
4. The fetcher builds the request via core.ts and wraps the raw response using envelope.ts.
5. The result bubbles back to meterDataforseoCall, which triggers trackDataforseoCost for billing.
6. The final unwrapped data (result.data) returns to the caller.

Implementation Examples

The following example demonstrates creating a metered client and fetching live SERP results:

import { createDataforseoClient } from "@/server/lib/dataforseo/client";
import { getOrganizationCustomer } from "@/server/billing/subscription";

async function fetchSerpExample() {
  const customer = await getOrganizationCustomer(); // Billing context
  const dfClient = createDataforseoClient(customer);

  // The call is automatically metered; you can optionally override the credit feature
  const serp = await dfClient.serp.live({
    target: "https://example.com",
    locationCode: 2840, // United States (DataForSEO Labs)
    languageCode: 1000, // English
    // creditFeature?: "rank_tracking" // optional
  });

  console.log("Organic results:", serp.organic);
}

For batch operations like rank-check tasks, the client supports explicit credit feature tagging:

import { createDataforseoClient } from "@/server/lib/dataforseo/client";

async function scheduleRankChecks(keywords: string[], projectId: string) {
  const dfClient = createDataforseoClient(customerContext);
  await dfClient.serp.rankCheckTaskPost({
    keywords,
    projectId,
    // The whole batch is charged as a single credit feature entry
    creditFeature: "rank_tracking",
  });
}

Summary

• Factory Pattern: createDataforseoClient in client.ts provides a type-safe, namespaced API for all DataForSEO endpoints.
• Performance Isolation: The SDK is lazy-loaded via loadDataforseoSections to prevent cold-start penalties.
• Billing Integration: The meter wrapper enforces usage-credit tracking in hosted mode via trackDataforseoCost.
• Standardized Errors: envelope.ts ensures consistent response handling and exposes billing metadata for charged-task errors.
• Runtime Adaptability: runtime-env.ts enables the same codebase to operate in both metered hosted and pass-through self-hosted modes.

Frequently Asked Questions

How does OpenSEO prevent the DataForSEO SDK from impacting server startup times?

OpenSEO uses dynamic imports in loadDataforseoSections to defer loading the ~3 MB SDK until the first API call. The SDK resides in src/server/lib/dataforseo/sections.ts, which is imported lazily and cached, keeping the initial server bundle lightweight.

What happens to API calls in self-hosted mode versus hosted mode?

In self-hosted mode, detected via isHostedServerAuthMode in runtime-env.ts, the meter wrapper bypasses billing logic entirely and forwards requests directly to DataForSEO. In hosted mode, every call passes through trackDataforseoCost to record expenses against the organization's credit pool.

How does the architecture handle billing for failed DataForSEO requests?

The envelope.ts module defines DataforseoChargedTaskError, which section fetchers throw when DataForSEO charges for a task that later fails. The meterDataforseoCall wrapper catches these errors and still invokes trackDataforseoCost to deduct credits, ensuring accurate billing even for unsuccessful API operations.

Which DataForSEO API sections are available through the OpenSEO client?

The client exposes seven primary namespaces: business, backlinks, keywords, domain, serp, labs, lighthouse, and aiSearch. Each maps to dedicated fetcher files (e.g., src/server/lib/dataforseo/serp.ts for SERP data) that handle endpoint-specific validation and response mapping.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}