# DataForSEO API Error Handling and Retry Logic in Open-SEO: A Deep Dive

> Explore Open-SEO's robust DataForSEO API error handling and retry logic. Our system automatically retries transient failures, classifies errors, and maintains billing accuracy for all tasks.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: deep-dive
- Published: 2026-07-27

---

**Open-SEO implements a resilient, multi-layered error handling pipeline for DataForSEO API calls that automatically retries transient failures, classifies errors by business impact, and preserves billing accuracy even when tasks fail after charging.**

The `every-app/open-seo` repository isolates all DataForSEO interactions behind a thin, well-tested abstraction layer. This design ensures that transient network issues and upstream service disruptions are handled gracefully while maintaining precise metering for billed operations. Understanding these patterns is essential for developers integrating third-party SEO data APIs into production workflows.

## Core Error Handling Architecture

The foundation of Open-SEO's DataForSEO integration resides in [`src/server/lib/dataforseo/core.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/core.ts). This module exports factory functions that wrap the native fetch API with authentication, timeouts, and retry logic.

### Authenticated Fetch with Built-in Retry

The `createAuthenticatedFetch` function constructs a request wrapper that enforces a **60-second timeout** and implements automatic retries for idempotent read operations. When the DataForSEO API returns a server error (`5xx`), the wrapper enters a retry loop governed by two constants:

- `DATAFORSEO_MAX_RETRIES = 2` (total attempts: initial + 2 retries)
- `DATAFORSEO_RETRY_BACKOFF_MS = 250` (linear back-off multiplier)

The back-off delay calculates as `250ms × (attempt + 1)`, producing intervals of 250ms, 500ms, and 750ms for successive retries. This bounded exponential strategy prevents thundering-herd problems while allowing brief upstream outages to resolve.

```typescript
import { serpApi } from "@/server/lib/dataforseo/core";

// Client automatically retries on 5xx with configured back-off
const api = serpApi();
const response = await api.task_post({
  target: "example.com",
  keyword: "seo tools",
  language_code: "en",
  location_code: 2840,
});

```

### Error Classification Strategy

Rather than throwing generic HTTP errors, the fetch factory accepts an optional **`DataforseoErrorClassifier`** callback. This function receives the HTTP status code, raw response body, and request path, returning a product-specific `AppError` or `null` to trigger default handling.

In [`src/server/lib/dataforseo/core.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/core.ts) (lines 90-106), unclassified errors fall back to a rich generic payload containing the full response context. This pattern allows different API consumers to map the same DataForSEO error to domain-specific exceptions—for example, converting a 403 response into `DATAFORSEO_AUTH_FAILED` or a rate limit into `RATE_LIMITED`.

```typescript
import type { DataforseoErrorClassifier } from "@/server/lib/dataforseo/core";
import { AppError } from "@/server/lib/errors";

const classifier: DataforseoErrorClassifier = (status, body, path) => {
  if (status === 403) {
    return new AppError("DATAFORSEO_AUTH_FAILED", "Invalid API credentials");
  }
  return null; // Defer to default error factory
};

const api = serpApi(classifier);

```

## Task-Level Validation and Billing Protection

Successful HTTP responses from DataForSEO return a JSON envelope containing one or more tasks, each with its own status code. The [`src/server/lib/dataforseo/envelope.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/envelope.ts) module provides `assertOk` and related utilities to validate these envelopes while safeguarding billing integrity.

### The assertOk Validation Pipeline

The `assertOk` function implements a five-stage validation protocol:

1. **Top-level status check**: Verifies the envelope's `status_code` equals `20000`; otherwise applies the error classifier or raises a generic exception.
2. **Task existence**: Confirms at least one task exists in the response array.
3. **Task-level status**: Validates the first task reports `20000` success.
4. **Billing-aware failures**: If a task failed but consumed credits (indicated by cost metadata), throws `DataforseoChargedTaskError` preserving the `cost` and `path` fields for metering.
5. **Empty results handling**: When `treatNoResultsAsEmpty` is true, converts "No Search Results" (`40501`) into an empty success rather than an error.

```typescript
import { assertOk } from "@/server/lib/dataforseo/envelope";

try {
  const task = assertOk(response, {
    classify: myClassifier,
    treatNoResultsAsEmpty: true,
  });
  // Process task.result...
} catch (e) {
  if (e instanceof DataforseoChargedTaskError) {
    // e.billing contains { path, costUsd } for metering
    await recordCharge(e.billing);
    throw e; // Re-throw or handle based on business logic
  }
}

```

### Billing Extraction and Charged Task Errors

The `tryBuildTaskBilling` helper (lines 70-84 in [`src/server/lib/dataforseo/envelope.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/envelope.ts)) validates envelope fields (`path`, `cost`, `result_count`) using Zod schemas. When `assertOk` detects a non-20000 task status but finds valid cost metadata, it constructs a `DataforseoChargedTaskError`. This ensures that failed operations are still metered accurately before error propagation, preventing revenue leakage.

### Invalid Field Diagnostics

For validation failures containing "Invalid Field: 'fieldName'" in the `status_message`, the `describeInvalidField` utility (lines 92-112) cross-references the original request payload (`task.data`). It appends the offending key-value pair to the error message, transforming opaque API errors into actionable debugging information.

## Retry Strategies for External Services

Beyond DataForSEO-specific handling, Open-SEO demonstrates consistent retry patterns for auxiliary services.

### Transient Upstream Retries (DataForSEO)

As implemented in [`src/server/lib/dataforseo/core.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/core.ts) (lines 80-88), the retry loop strictly limits retries to `5xx` status codes. The linear back-off calculation (`DATAFORSEO_RETRY_BACKOFF_MS × (attempt + 1)`) keeps latency predictable while the 60-second request timeout provides an absolute ceiling. This protects against brief downstream outages without exposing users to indefinite hanging requests.

### Rate Limit Handling for Autumn Billing

The billing integration in [`src/serverFunctions/billing.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/billing.ts) handles HTTP 429 (Too Many Requests) from the Autumn service using a distinct strategy:

- **Maximum attempts**: `AUTUMN_MAX_RETRIES = 3`
- **Respect Retry-After**: Parses the header value, capping delay at `AUTUMN_MAX_RETRY_DELAY_MS = 5000ms`
- **Progressive fallback**: When the header is missing, applies `AUTUMN_RETRY_BACKOFF_MS × (attempt + 1)`

This pattern demonstrates how Open-SEO adapts retry logic to specific API contracts—honoring server-specified delays when available while maintaining defensive defaults.

```typescript
// Simplified excerpt from fetchAutumnEventsPage
if (resp.status === 429 && attempt < AUTUMN_MAX_RETRIES) {
  const retryAfter = Number(resp.headers.get("Retry-After"));
  const delay = Number.isFinite(retryAfter)
    ? Math.min(retryAfter * 1000, AUTUMN_MAX_RETRY_DELAY_MS)
    : AUTUMN_RETRY_BACKOFF_MS * (attempt + 1);
  await new Promise(r => setTimeout(r, delay));
  continue;
}

```

## Database Resilience Patterns

While not specific to DataForSEO, the repository's [`src/db/pg/retry.ts`](https://github.com/every-app/open-seo/blob/main/src/db/pg/retry.ts) module illustrates the broader architectural commitment to resilience. The `withQueryRetries` utility wraps PostgreSQL queries, automatically retrying on transient connection errors (`ECONNRESET`) with jitter to prevent thundering-herd scenarios. This consistency across external HTTP APIs and internal database connections ensures system-wide stability.

## Summary

- **Automatic retries**: DataForSEO API calls retry on `5xx` errors up to 2 times with linear back-off (250ms increments) and a 60-second timeout.
- **Structured error classification**: The `DataforseoErrorClassifier` pattern allows custom mapping of HTTP status codes to domain-specific `AppError` types.
- **Billing protection**: `assertOk` validates task-level status codes and throws `DataforseoChargedTaskError` for failed tasks that consumed credits, preserving cost metadata.
- **Diagnostic enrichment**: Invalid field errors include the original request payload values for rapid debugging.
- **Service-specific strategies**: Rate-limited billing endpoints (Autumn) respect `Retry-After` headers with distinct constants (`AUTUMN_MAX_RETRIES = 3`).

## Frequently Asked Questions

### How does Open-SEO handle DataForSEO API rate limits?

Open-SEO primarily relies on DataForSEO's high rate limits and stable infrastructure; however, for the Autumn billing service (which uses distinct rate limiting), the implementation in [`src/serverFunctions/billing.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/billing.ts) specifically handles HTTP 429 responses. It parses the `Retry-After` header and delays subsequent requests up to a maximum of 5000ms, retrying up to 3 times before failing.

### What happens when a DataForSEO task fails after billing?

When `assertOk` detects a task with a non-20000 status code that includes billing metadata (`cost`, `path`), it throws `DataforseoChargedTaskError`. This specialized error type preserves the billing information, allowing higher-level services to record the charge via `meterDataforseoCall` before deciding whether to propagate the error or attempt recovery.

### Can I customize which DataForSEO errors are considered retryable?

Yes. While the built-in retry logic in `createAuthenticatedFetch` automatically retries only `5xx` server errors, you can pass a custom `DataforseoErrorClassifier` to the client factory. This classifier receives the HTTP status, body, and path, allowing you to map specific error conditions (like 403 authentication failures) to appropriate `AppError` types before the retry logic evaluates them.

### What is the maximum latency for a DataForSEO API call including retries?

The maximum latency is bounded by the 60-second request timeout configured in `createAuthenticatedFetch`. With `DATAFORSEO_MAX_RETRIES = 2` and back-off delays of 250ms and 500ms, the worst-case scenario adds 750ms of retry overhead to the initial request duration, well within the timeout window.