How OpenSEO Handles DataForSEO API Calls with Retry Logic and Timeouts

OpenSEO implements DataForSEO API resilience through a centralized wrapper in src/server/lib/dataforseo/core.ts that combines AbortController-based timeouts with selective exponential backoff retry logic—retrying only idempotent read operations while strictly avoiding retries on billable task-post endpoints.

The every-app/open-seo repository provides a production-ready TypeScript integration with the DataForSEO API. Rather than calling the dataforseo-client SDK directly, all requests flow through a protective abstraction layer that handles transient failures, prevents double-billing, and enforces hard deadlines on every outbound call.

Why Centralized Request Handling Matters

DataForSEO's infrastructure occasionally returns transient 5xx errors during maintenance windows or traffic spikes. Without careful handling, these failures can:

• Corrupt user workflows by surfacing raw SDK errors
• Double-charge accounts if task-creation endpoints are naively retried
• Hang indefinitely without network-level timeouts

The core.ts module solves these problems through context-aware retry policies and mandatory request timeouts.

Core Implementation: Timeouts and Abort Controllers

Every DataForSEO request in OpenSEO is wrapped with a fresh AbortController instance. In src/server/lib/dataforseo/core.ts, the timeout logic creates a deadline that cannot be bypassed:

// src/server/lib/dataforseo/core.ts (simplified)
export async function fetchWithRetry<T>(fn: () => Promise<T>, opts = {}) {
  const { retries = 3, timeout = 30_000 } = opts;
  for (let attempt = 0; attempt <= retries; ++attempt) {
    const controller = new AbortController();
    const timer = setTimeout(() => controller.abort(), timeout);
    try {
      const result = await fn({ signal: controller.signal });
      clearTimeout(timer);
      return result;
    } catch (err) {
      clearTimeout(timer);
      // Retry logic evaluated here...
      throw err;
    }
  }
}

Key timeout characteristics:

• Default deadline: 30 seconds (DEFAULT_TIMEOUT_MS = 30_000)
• Configurable per-call: Override via opts.timeout for long-running batch operations
• Guaranteed cleanup: clearTimeout(timer) prevents memory leaks on both success and failure paths
• Abort signal propagation: Passed through to the underlying SDK call for true cancellation

Selective Retry Logic: Idempotent vs. Billable Endpoints

The most critical design decision in OpenSEO's DataForSEO handling is endpoint-aware retry behavior. The system distinguishes between safe, idempotent reads and dangerous, billable writes.

Read-Only Operations: Exponential Backoff Retry

Endpoints that fetch existing data (SERP results, keyword rankings, competitor analysis) receive full retry protection:

Retry Parameter	Value
Maximum attempts	4 total (initial + 3 retries)
Backoff strategy	Exponential: 100ms → 200ms → 400ms
Retryable status codes	502, 503, 504, 429 (rate limit)

These calls are handled by helpers like serpApi in src/server/lib/dataforseo/serp.ts:

// src/server/lib/dataforseo/serp.ts
import { fetchWithRetry } from './core';

export async function serpApi(client: DataForSeoClient, payload: SerpPayload) {
  // Marked as idempotent → retries enabled
  return fetchWithRetry(
    () => client.serpApi.googleOrganicTaskGet(payload),
    { retries: 3, idempotent: true }
  );
}

Task-Post Endpoints: No Retry Protection

Endpoints that create new tasks (taskPost, taskUpload) are never retried regardless of error type. This prevents double-charging when a 5xx occurs after the server has already accepted the request.

In src/server/lib/dataforseo/business.ts, these calls bypass fetchWithRetry entirely or invoke it with retries: 0:

// src/server/lib/dataforseo/business.ts
export async function createBacklinksTask(
  client: DataForSeoClient,
  payload: TaskPostPayload
) {
  // Explicitly disable retry for billable operation
  return fetchWithRetry(
    () => client.backlinksApi.taskPost(payload),
    { retries: 0 } // CRITICAL: Prevents double-billing
  );
  // Alternative: Direct SDK call with timeout wrapper only
}

Error Classification and Response Normalization

Retry decisions depend on accurate error classification. OpenSEO's src/server/lib/dataforseo/envelope.ts provides the DataforseoError type hierarchy and isTransient() utility:

// src/server/lib/dataforseo/envelope.ts
export function isTransient(error: unknown): boolean {
  if (!(error instanceof DataforseoError)) return false;
  // Network errors and specific HTTP status codes
  return (
    error.code === 'ECONNRESET' ||
    error.code === 'ETIMEDOUT' ||
    (error.status >= 500 && error.status < 600) ||
    error.status === 429
  );
}

export function assertOk<T>(response: ApiResponse<T>): T {
  if (response.status_code !== 20000) { // DataForSEO success code
    throw new DataforseoError(response.status_message, response.status_code);
  }
  return response.tasks;
}

The envelope.ts module ensures that SDK responses are validated consistently before retry logic evaluates them.

Complete Usage Example

Here's how OpenSEO's DataForSEO abstraction is used in practice, combining client initialization, retry-enabled reads, and protected task creation:

import { createDataforseoClient } from '@/server/lib/dataforseo/client';
import { serpApi } from '@/server/lib/dataforseo/core';
import { createBacklinksTask } from '@/server/lib/dataforseo/business';

// Initialize once per request or use singleton pattern
const client = createDataforseoClient({
  username: process.env.DATAFORSEO_USERNAME!,
  password: process.env.DATAFORSEO_PASSWORD!,
});

// Example 1: Fetch organic SERP results (with retry/timeout)
async function fetchRankings(keyword: string, domain: string) {
  const result = await serpApi(client, {
    keyword,
    target: domain,
    location_code: 2840,  // New York, USA
    language_code: 'en',
  });
  return result.tasks?.[0]?.result?.[0]?.items ?? [];
}

// Example 2: Start new backlink analysis (no retry, timeout protected)
async function startBacklinkAnalysis(projectId: string) {
  try {
    const task = await createBacklinksTask(client, { project_id: projectId });
    return task.tasks?.[0]?.id; // Task ID for polling
  } catch (err) {
    // Handle timeout or error without retry risk
    if (err.name === 'AbortError') throw new Error('Request timed out');
    throw err;
  }
}

Architecture Overview: Request Flow

┌─────────────────┐     ┌─────────────────────────────┐     ┌─────────────────┐
│  Application    │────▶│  src/server/lib/dataforseo/ │────▶│  dataforseo-    │
│  Code           │     │  core.ts: fetchWithRetry()  │     │  client SDK     │
└─────────────────┘     └─────────────────────────────┘     └─────────────────┘
                               │        ▲
                               ▼        │
                        ┌──────────────┴────────┐
                        │  AbortController        │
                        │  (timeout enforcement)  │
                        │                         │
                        │  Retry loop (conditional) │
                        │  • Idempotent reads: YES  │
                        │  • Task-post: NO          │
                        └─────────────────────────┘

Key Source Files

File	Responsibility
src/server/lib/dataforseo/core.ts	Central fetchWithRetry() implementation with AbortController timeouts and backoff logic
src/server/lib/dataforseo/envelope.ts	Response normalization, DataforseoError types, isTransient() classification
src/server/lib/dataforseo/client.ts	Factory for configured SDK instances with injected wrappers
src/server/lib/dataforseo/serp.ts	SERP-specific helpers (read-only, retry-enabled)
src/server/lib/dataforseo/business.ts	Task-post endpoints (retry explicitly disabled)

Summary

• Timeouts are mandatory: Every DataForSEO request carries an AbortController with a 30-second default deadline
• Retry is conditional: Only idempotent read operations receive exponential backoff retry; billable task-posts never retry
• Implementation centers on core.ts: The fetchWithRetry() function in src/server/lib/dataforseo/core.ts orchestrates all protective behavior
• Error classification matters: src/server/lib/dataforseo/envelope.ts provides the isTransient() logic that gates retry decisions
• Double-billing protection is architectural: Task-post endpoints in business.ts explicitly set retries: 0

Frequently Asked Questions

What is the default timeout for DataForSEO API calls in OpenSEO?

The default timeout is 30 seconds (30,000 ms), defined as DEFAULT_TIMEOUT_MS in src/server/lib/dataforseo/core.ts. This can be overridden per-call via the timeout option in fetchWithRetry().

Why doesn't OpenSEO retry task-post endpoints?

Task-post endpoints create billable jobs on DataForSEO's infrastructure. A retry after a transient 5xx could create duplicate tasks and double-charge the account. OpenSEO's business.ts module explicitly disables retries for these operations to prevent financial risk.

How does OpenSEO distinguish between retryable and non-retryable errors?

The isTransient() function in src/server/lib/dataforseo/envelope.ts classifies errors by status code (502, 503, 504, 429) and network error codes (ECONNRESET, ETIMEDOUT). Additionally, the idempotent flag passed to fetchWithRetry() determines whether retry logic executes even for transient errors.

Can I adjust the retry count or backoff timing?

Yes. The fetchWithRetry() function accepts opts.retries (default: 3) and the backoff calculation can be customized in core.ts. The current implementation uses backoff(attempt) = 100 * 2^attempt milliseconds, producing 100ms, 200ms, 400ms delays across three retry attempts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}