# How the DataForSEO Client Handles API Request Batching and Error Recovery in Open SEO

> Discover how the DataForSEO client in Open SEO efficiently batches API requests and recovers from errors. Learn about charged errors versus validation failures for optimized credit usage.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: internals
- Published: 2026-07-30

---

**The DataForSEO client in Open SEO uses a meter wrapper that automatically batches bulk API requests into compliant chunk sizes while centralizing error handling to distinguish between charged errors (which still consume credits) and validation failures (which do not).**

The Open SEO repository integrates with the DataForSEO API through a thin abstraction layer designed to manage high-volume data operations efficiently. This implementation tackles two critical challenges inherent to third-party SEO data providers: optimizing expensive network calls through intelligent batching and ensuring accurate billing attribution even when requests fail. Understanding how the DataForSEO client handles API request batching and error recovery reveals a robust pattern for managing metered external APIs in production TypeScript applications.

## Lazy Loading and the Meter Wrapper Architecture

The client architecture prioritizes cold-start performance by deferring the loading of the ~3 MB `dataforseo-client` SDK until the first actual API invocation. In [`src/server/lib/dataforseo/client.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/client.ts), the `loadDataforseoSections()` function initializes the heavy dependency only when needed, keeping serverless function startup times minimal.

The core abstraction is the `meter()` wrapper function, which transforms raw DataForSEO section methods into metered operations. When `createDataforseoClient()` is invoked with a customer context, it constructs an object where every leaf method passes through `meterDataforseoCall`. This wrapper receives three key parameters: the customer billing context, a picker function targeting the real SDK method, and an optional default credit feature identifier.

Execution flow diverges based on deployment mode. If the server runs in hosted mode (where usage credits must be tracked), `meterDataforseoCall` intercepts the request to apply billing logic; otherwise, it returns raw data directly without overhead.

## Batching Strategies for High-Volume Endpoints

The client implements endpoint-specific batching logic to respect DataForSEO's payload limits while maximizing throughput. Large datasets are automatically sliced into compliant chunks, with each chunk executed as an independent API call.

### Keyword Metrics Batching (700 Keywords Per Request)

For keyword volume and metrics retrieval, the client defines a `KEYWORD_METRICS_BATCH_SIZE` constant set to **700** in [`src/server/lib/dataforseo/keyword-metrics.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/keyword-metrics.ts). The public helper `fetchKeywordMetricsForList` slices input arrays exceeding this threshold into chunks, invoking the underlying endpoint for each segment sequentially.

This approach ensures that requests containing 1,500 keywords, for example, automatically split into two batches of 700 and one batch of 100. Each batch maintains the same location and language parameters, with results aggregated before returning to the caller.

### Rank Check Task Batching (100 Tasks Per Request)

The rank monitoring functionality leverages DataForSEO's ability to queue multiple tasks in a single POST operation. According to the implementation in [`src/server/lib/dataforseo/client.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/client.ts), the `rankCheckTaskPost` method supports posting **up to 100 queued rank-check tasks** simultaneously.

Because DataForSEO bills the entire `task_post` request as a single unit regardless of task count, the meter wrapper charges exactly one credit feature for the complete batch. This design optimizes cost efficiency when scheduling large-scale rank tracking operations across multiple keywords or domains.

## Error Recovery and Credit Tracking Mechanisms

The `meterDataforseoCall` function serves as the central error handling chokepoint, implementing sophisticated logic to differentiate between billable failures and client-side validation errors.

### Distinguishing Charged vs. Validation Errors

When exceptions occur inside `meterDataforseoCall`, the wrapper inspects the error type defined in [`src/server/lib/dataforseo/envelope.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/envelope.ts). If the error is a `DataforseoChargedTaskError`, indicating that DataForSEO has already billed the request despite the failure, the error is logged and credit usage is still recorded via `trackDataforseoCost`.

Conversely, for validation-type errors where `error.isInvalidField` is true and `error.billing.costUsd <= 0`, the wrapper throws a clean `AppError` with code `"VALIDATION_ERROR"`. This allows calling code to handle malformed inputs without consuming user credits, distinguishing between "user made a mistake" and "provider failed after charging."

### Unified Credit Tracking with trackDataforseoCost

Regardless of success or the specific error type, `trackDataforseoCost` is invoked to persist usage data. This function records the spent credits, provider identifier (`dataforseo`), API path, and cache status. The implementation ensures that even failed batches that consumed resources upstream are correctly attributed to the customer, maintaining billing integrity across all scenarios.

After successful execution and credit bookkeeping, the wrapper returns `result.data` (the unwrapped payload) to the original caller, abstracting away the metering complexity.

## Implementation Examples

```typescript
// Fetching keyword metrics for 1500 keywords (auto-batched into chunks of 700)
import { createDataforseoClient } from "@/server/lib/dataforseo/client";
import { getOrganizationCustomer } from "@/server/billing/subscription";

async function getMetrics(keywords: string[]) {
  const customer = await getOrganizationCustomer(/* org context */);
  const client = createDataforseoClient(customer);
  
  // Automatically splits into 2 batches of 700 + 1 batch of 100
  const rows = await client.keywords.adsSearchVolume({
    keywords,
    locationCode: 2840,          // United States
    locationName: "New York",
    languageCode: "en",
    creditFeature: "rank_tracking",
  });
  
  return rows; // Normalized rows aggregated from all batches
}

```

```typescript
// Posting a batch of rank-check tasks (up to 100 tasks per call)
import { createDataforseoClient } from "@/server/lib/dataforseo/client";

async function queueRankChecks(tasks: RankCheckTask[]) {
  const client = createDataforseoClient(/* billing customer */);
  
  // Single credit charged for the entire batch of up to 100 tasks
  await client.serp.rankCheckTaskPost({
    tasks,
    creditFeature: "rank_tracking",
  });
}

```

## Summary

- **Lazy initialization** keeps cold-starts fast by loading the 3 MB DataForSEO SDK only on first use via `loadDataforseoSections()`.
- **Automatic batching** splits keyword lists into 700-item chunks and rank-check tasks into 100-task batches to respect API limits.
- **Meter wrapper** (`meterDataforseoCall`) centralizes all API calls to enforce credit tracking and error classification.
- **Error differentiation** distinguishes between `DataforseoChargedTaskError` (billable failures) and validation errors (non-billable), ensuring users only pay for actual provider usage.
- **Unified tracking** via `trackDataforseoCost` records every transaction, including failed requests that consumed credits, maintaining accurate billing records.

## Frequently Asked Questions

### What is the maximum number of keywords that can be processed in a single API call?

The DataForSEO client in Open SEO automatically batches keyword metrics requests into chunks of **700 keywords** per API call. If you submit a list of 1,500 keywords, the `fetchKeywordMetricsForList` function in [`src/server/lib/dataforseo/keyword-metrics.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/keyword-metrics.ts) splits this into three separate requests (700, 700, and 100) and aggregates the results transparently.

### How does the client handle errors where DataForSEO has already charged for the request?

When a `DataforseoChargedTaskError` occurs—indicating the provider billed the request before failing—the `meterDataforseoCall` wrapper catches the exception, logs the failure, and still invokes `trackDataforseoCost` to record the spent credits. The error is then re-thrown so the application can handle the failure while maintaining accurate billing records.

### What happens to validation errors that don't consume API credits?

For validation errors where `error.isInvalidField` is true and no cost was incurred (`error.billing.costUsd <= 0`), the wrapper throws a clean `AppError` with the code `"VALIDATION_ERROR"`. This prevents the credit tracking system from recording a charge, ensuring users do not pay for malformed requests or invalid parameters.

### Why does the rank-check task endpoint support exactly 100 tasks per batch?

DataForSEO's SERP API accepts up to 100 queued tasks in a single `task_post` request and bills the entire operation as one unit. The Open SEO client respects this limit in [`src/server/lib/dataforseo/client.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/client.ts) to optimize cost efficiency, charging a single credit feature for the complete batch rather than individual charges per task.