# DataForSEO Client Architecture for Error Handling and Retries in Open-SEO

> Discover the DataForSEO client architecture in Open-SEO. Learn how network transport, response normalization, and the Autumn engine handle errors and implement retries with exponential back-off.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: architecture
- Published: 2026-08-19

---

**The every-app/open-seo repository implements a four-layer DataForSEO client architecture that isolates network transport in [`core.ts`](https://github.com/every-app/open-seo/blob/main/core.ts), normalizes API responses through [`envelope.ts`](https://github.com/every-app/open-seo/blob/main/envelope.ts), and delegates retry logic to the Autumn engine ([`autumn.ts`](https://github.com/every-app/open-seo/blob/main/autumn.ts)) to handle transient 5xx/429 errors with exponential back-off while strictly preventing retries of charged task errors via `DataforseoChargedTaskError`.**

The every-app/open-seo codebase provides a production-grade TypeScript client for the DataForSEO API that prioritizes reliability and cost safety. This article examines the DataForSEO client architecture for error handling and retries, detailing how the system separates transient network failures from provider-specific charged errors to prevent double-billing while maximizing request success rates.

## The Four-Layer Architecture

The client is organized into distinct layers, each with a single responsibility regarding error propagation and recovery.

### 1. Core HTTP Wrapper ([`core.ts`](https://github.com/every-app/open-seo/blob/main/core.ts))

The foundation resides in [`src/server/lib/dataforseo/core.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/core.ts), which manages raw HTTP transport to `api.dataforseo.com`. This module adds required authentication headers, performs the fetch operation, and parses JSON responses. It detects transient HTTP errors—specifically **5xx server errors** and **429 rate-limit responses**—and extracts `Retry-After` headers when present. When transient failures occur, it throws `TransientDataforseoError` signals that the upstream retry engine consumes.

### 2. Envelope Parsing and Error Classification ([`envelope.ts`](https://github.com/every-app/open-seo/blob/main/envelope.ts))

Located at [`src/server/lib/dataforseo/envelope.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/envelope.ts), this layer normalizes DataForSEO's envelope structure (`task_post`, `task_get`, `task_status`) into typed results. It distinguishes between **transient** failures (network issues, rate limits) and **charged** failures (invalid parameters that still consume API credits). When the envelope contains a charged failure status code such as `20002` (Invalid field), the parser throws `DataforseoChargedTaskError`. This error class explicitly signals that the operation should **not** be retried, protecting users from accidental double-billing.

### 3. The Autumn Retry Engine ([`autumn.ts`](https://github.com/every-app/open-seo/blob/main/autumn.ts))

The retry logic lives in [`src/server/billing/autumn.ts`](https://github.com/every-app/open-seo/blob/main/src/server/billing/autumn.ts), a generic retry utility used across the codebase. The Autumn engine wraps request functions in a configurable loop that respects exponential back-off with jitter. It retries only idempotent operations—primarily `GET` requests and task polling—against a configurable allowlist of status codes (default: `["429", "500", "502", "503", "504"]`). The engine honors `Retry-After` headers when available and defaults to exponential back-off otherwise, capping attempts at `maxAttempts` (default `3`) to bound latency.

### 4. Service-Level APIs ([`serp.ts`](https://github.com/every-app/open-seo/blob/main/serp.ts), [`labs.ts`](https://github.com/every-app/open-seo/blob/main/labs.ts), etc.)

Concrete implementations such as [`src/server/lib/dataforseo/serp.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/serp.ts) compose the lower layers into domain-specific methods like `fetchSerpOrganic()`. These modules import the core wrapper, envelope parser, and Autumn helper, ensuring every DataForSEO request automatically inherits uniform error handling and retry semantics without duplicating logic across the codebase.

## Error Handling Flow

The following sequence illustrates how a SERP request moves through the architecture when encountering failures:

1. **Client Initialization** – `createDataforseoClient()` (in [`src/server/lib/dataforseo/client.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/client.ts)) instantiates a client with API credentials and base URL configuration.

2. **Request Execution** – Service modules invoke `core.request()`, which performs the HTTP fetch. On **5xx** or **429** responses, it throws `TransientDataforseoError`.

3. **Retry Evaluation** – The Autumn engine (`autumn.check()`) intercepts transient errors and evaluates them against `retryCodes`. Valid transient errors trigger exponential back-off; charged errors like `DataforseoChargedTaskError` bubble through uncaught by the retry loop.

4. **Envelope Validation** – For successful HTTP responses (200 OK), [`envelope.ts`](https://github.com/every-app/open-seo/blob/main/envelope.ts) validates the DataForSEO payload. Invalid parameters result in `DataforseoChargedTaskError`, which immediately surface to the caller without retry attempts.

5. **Workflow Consumption** – Higher-level workflows (e.g., [`src/server/workflows/RankCheckWorkflow.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/RankCheckWorkflow.ts)) catch these errors. Charged failures mark the step as failed with user-facing messages; surviving transient errors after exhaustion of `maxAttempts` are logged as critical upstream failures.

## Key Implementation Details

### Transient vs. Charged Error Semantics

The architecture strictly separates error types to balance reliability against cost:

- **TransientDataforseoError**: Represents temporary infrastructure issues (network timeouts, 5xx, rate limits). These are safe to retry and are handled by Autumn.
- **DataforseoChargedTaskError**: Represents semantic failures (invalid location codes, malformed parameters). DataForSEO bills these requests regardless of failure, so the client immediately surfaces them to prevent duplicate charges.

### Retry Configuration Parameters

The Autumn engine accepts per-call overrides:

- `retryCodes`: Array of HTTP status codes eligible for retry (default includes 429, 500, 502, 503, 504).
- `maxAttempts`: Total execution attempts including the initial call (default `3`).
- `initialDelayMs`: Base delay for exponential back-off calculation.

## Practical Code Examples

### Creating a DataForSEO Client

```typescript
import { createDataforseoClient } from '@/server/lib/dataforseo';

const client = createDataforseoClient({
  apiKey: process.env.DATAFORSEO_API_KEY!,
  baseUrl: 'https://api.dataforseo.com',
});

```

### Executing a SERP Request with Automatic Retries

```typescript
import { fetchSerpOrganic } from '@/server/lib/dataforseo/serp';

async function getOrganicResults(url: string, location: string) {
  // Automatically retries transient 5xx/429 errors via Autumn
  const payload = await fetchSerpOrganic(client, {
    target: url,
    location_code: location,
  });
  return payload;
}

```

### Handling Charged and Transient Errors

```typescript
import { DataforseoChargedTaskError } from '@/server/lib/dataforseo/envelope';

try {
  const results = await getOrganicResults('https://example.com', 'us');
} catch (err) {
  if (err instanceof DataforseoChargedTaskError) {
    // Charged failure: do not retry, surface to user
    logger.warn('Invalid DataForSEO parameters: %s', err.message);
    throw new AppError('INVALID_LOCATION', err.message);
  }
  // Transient errors already retried by Autumn;
  // reaching here indicates exhaustion of maxAttempts
  throw err;
}

```

### Custom Retry Configuration

```typescript
import { autumn } from '@/server/billing/autumn';

await autumn.check(
  async () => fetchSerpOrganic(client, { target: url, location_code: 'us' }),
  {
    retryCodes: ['429', '500'],
    maxAttempts: 5,
    initialDelayMs: 200,
  }
);

```

## Summary

- The client architecture separates concerns across four layers: **Core** ([`core.ts`](https://github.com/every-app/open-seo/blob/main/core.ts)), **Envelope** ([`envelope.ts`](https://github.com/every-app/open-seo/blob/main/envelope.ts)), **Retry Engine** ([`autumn.ts`](https://github.com/every-app/open-seo/blob/main/autumn.ts)), and **Service APIs** ([`serp.ts`](https://github.com/every-app/open-seo/blob/main/serp.ts)).
- **Transient errors** (5xx, 429) trigger exponential back-off via the Autumn engine, respecting `Retry-After` headers and `maxAttempts` limits.
- **Charged errors** (`DataforseoChargedTaskError`) bypass retry logic entirely to prevent double-billing for invalid requests.
- All DataForSEO interactions inherit uniform error handling through the service-level abstractions, ensuring consistent behavior across SERP, Labs, Backlinks, and Google Ads endpoints.

## Frequently Asked Questions

### How does the Open-SEO client prevent double-billing on failed requests?

The architecture uses `DataforseoChargedTaskError` thrown by [`src/server/lib/dataforseo/envelope.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/envelope.ts) to identify failures that consume API credits (e.g., invalid location codes). The Autumn retry engine in [`src/server/billing/autumn.ts`](https://github.com/every-app/open-seo/blob/main/src/server/billing/autumn.ts) only catches `TransientDataforseoError`, ensuring charged failures immediately surface to the caller without retry attempts.

### What transient errors trigger automatic retries?

The client retries HTTP status codes `429`, `500`, `502`, `503`, and `504` by default. These are caught in [`src/server/lib/dataforseo/core.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/core.ts) and converted to `TransientDataforseoError`, which the `autumn.check()` wrapper retries using exponential back-off with jitter, honoring any `Retry-After` headers provided by DataForSEO.

### Can I adjust retry behavior for specific API calls?

Yes. While the service modules ([`serp.ts`](https://github.com/every-app/open-seo/blob/main/serp.ts), etc.) use sensible defaults, you can invoke `autumn.check()` directly with a custom configuration object specifying `retryCodes`, `maxAttempts`, and `initialDelayMs` to override the global retry policy for individual requests.

### Where is the retry logic implemented if I need to debug timeout issues?

The central retry implementation resides in [`src/server/billing/autumn.ts`](https://github.com/every-app/open-seo/blob/main/src/server/billing/autumn.ts). This file manages the retry loop, back-off calculation, and error classification. For DataForSEO-specific error translation (mapping HTTP responses to charged vs. transient errors), inspect [`src/server/lib/dataforseo/envelope.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/envelope.ts) and [`src/server/lib/dataforseo/core.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/core.ts).