OpenSEO Architecture and External API Integration: A Deep Dive into the Layered TypeScript Design

OpenSEO uses a TanStack-Server-Function → Service → Repository pattern to orchestrate calls to DataForSEO, Google Search Console, and Google Analytics 4, wrapping each external request in a billing-aware metering layer.

OpenSEO is a modern full-stack TypeScript application in the every-app/open-seo repository that cleanly separates routing logic from business intelligence and persistence. Its OpenSEO architecture revolves around three distinct layers that handle everything from HTTP endpoints to credit-metered API calls, ensuring that external provider costs are accurately tracked against organization quotas.

Three-Layer Architecture Pattern

The codebase in src/ follows a strict layered approach that keeps external API concerns isolated from transport and storage logic.

Router and Server Functions Layer

This layer defines HTTP endpoints and request validation. Files in src/server/—such as src/server/workflows/RankCheckWorkflow.ts—expose TanStack Server Functions that act as the entry point for client requests. These functions validate inputs and delegate to service-layer clients without containing direct database or third-party API logic.

Service Layer

Located in src/server/lib/, this layer contains the business logic that orchestrates external API calls. It handles billing metering, data transformation, and error handling. For example, the DataForSEO integration lives entirely within src/server/lib/dataforseo/, while Google services are managed in src/server/lib/gsc.ts and src/server/lib/ga4.ts.

Repository Layer

The persistence layer in src/db/—exemplified by src/db/schema.ts—provides direct access to SQLite or Postgres via Drizzle ORM. This layer is only accessed by services or workflows, never directly by router functions, ensuring data consistency across the OpenSEO external API integration workflows.

External API Integration Strategy

OpenSEO communicates with three major third-party services, each abstracted behind a typed client and a unified billing metering system.

DataForSEO Integration

The DataForSEO client represents the most complex integration, handling SERP, keyword, backlink, lighthouse, and AI-search data. The implementation in src/server/lib/dataforseo/client.ts creates a scoped client per billing customer:

// src/server/lib/dataforseo/client.ts
export function createDataforseoClient(customer: BillingCustomerContext) {
  return {
    serp: {
      rankCheck: meter(customer, (s) => s.fetchRankCheckSerp, "rank_tracking"),
    },
    // …other sections
  } as const;
}

To optimize cold-start performance, the DataForSEO SDK (~3MB) is lazy-loaded via loadDataforseoSections(). The actual fetchers for specific endpoints—such as fetchRankCheckSerp in src/server/lib/dataforseo/serp.ts and Google Ads data in src/server/lib/dataforseo/google-ads.ts—are only imported when the first request executes.

Google Search Console OAuth Flow

Google Search Console (GSC) integration relies on OAuth tokens stored in user sessions. Constants defining the provider ID and scopes reside in src/shared/gsc.ts:

// src/shared/gsc.ts
export const GSC_OAUTH_PROVIDER_ID = "google-search-console";
export const GSC_OAUTH_SCOPES = [
  "https://www.googleapis.com/auth/webmasters.readonly",
];

The server-side implementation in src/server/lib/gsc.ts retrieves these tokens using the provider configuration from src/lib/auth-config.ts and instantiates the Google Webmasters API client with the user's credentials.

Google Analytics 4 Implementation

GA4 follows an identical pattern to GSC. Shared constants live in src/shared/ga4.ts, while the execution logic sits in src/server/lib/ga4.ts. The client uses the Google Analytics Data API (analyticsDataClient) to fetch traffic, event, and conversion metrics, respecting the OAuth token stored alongside the GSC credentials.

Billing and Credit Metering

Every external API call passes through a common billing metering layer via the meter helper defined in billing utilities. The meterDataforseoCall function (lines 137-188 in the envelope implementation) records usage against specific credit features like "rank_tracking" or "local_seo". When operating in hosted mode, credits are deducted from the organization's monthly quota; self-hosted instances bypass this deduction entirely.

Data Flow Example: Rank Check Workflow

The RankCheckWorkflow in src/server/workflows/RankCheckWorkflow.ts demonstrates how these layers interact to process a SERP ranking request:

1. Endpoint entry: The TanStack Server Function receives input and calls runRankCheck().
2. Client instantiation: The workflow invokes createDataforseoClient(customer) to obtain a client scoped to the requesting organization's billing context.
3. Lazy execution: Calling dfClient.serp.rankCheck(input) triggers loadDataforseoSections() to import the SDK on demand.
4. Metered API call: The underlying fetchRankCheckSerp function executes within meterDataforseoCall, which tracks the operation's cost and deducts credits.
5. Caching: Results are optionally cached via the R2 cache wrapper in src/server/lib/r2-cache.ts before returning to the client.

// src/server/workflows/RankCheckWorkflow.ts
export async function runRankCheck(
  customer: BillingCustomerContext, 
  input: RankCheckInput
) {
  const dfClient = createDataforseoClient(customer);
  const serpResult = await dfClient.serp.rankCheck(input);
  // Result is metered and cost-tracked automatically
  return serpResult;
}

Summary

• OpenSEO implements a TanStack-Server-Function → Service → Repository pattern that separates HTTP handling, business logic, and database access.
• DataForSEO integration uses lazy-loaded sections (loadDataforseoSections) to avoid importing the 3MB SDK until necessary, with scoped clients created per billing customer in src/server/lib/dataforseo/client.ts.
• Google Search Console and GA4 integrations rely on shared OAuth constants in src/shared/ and server-side clients in src/server/lib/ that respect session-stored tokens.
• All external calls are wrapped in a metering layer (meterDataforseoCall) that records usage against credit features like "rank_tracking" for hosted billing scenarios.
• The architecture supports optional R2 caching (src/server/lib/r2-cache.ts) to reduce redundant external API costs.

Frequently Asked Questions

How does OpenSEO handle the large DataForSEO SDK bundle size?

OpenSEO uses dynamic imports via loadDataforseoSections() to lazy-load the approximately 3MB DataForSEO SDK only when an API request actually requires it. This prevents the large dependency from bloating the initial server startup or cold-start times.

Where is the Google Search Console OAuth configuration defined?

The OAuth provider ID and scopes are exported from src/shared/gsc.ts as GSC_OAUTH_PROVIDER_ID and GSC_OAUTH_SCOPES. These constants are consumed by the authentication configuration in src/lib/auth-config.ts and used by the server-side client in src/server/lib/gsc.ts to authenticate requests with stored user tokens.

What happens to external API calls when self-hosting OpenSEO?

According to the source in billingCreditFeatures.ts, the metering system detects the deployment mode. In self-hosted mode, calls to external APIs bypass the credit deduction logic, allowing unlimited usage without checking against a monthly organization quota, though the metering instrumentation still records the usage for analytics.

Which file manages the database schema for caching API results?

The persistence layer uses Drizzle ORM defined in src/db/schema.ts, while the R2-compatible cache wrapper for external API responses is implemented in src/server/lib/r2-cache.ts. This cache sits between the service layer and external APIs to reduce redundant calls and costs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}