Open‑SEO Performance: 8 Critical Considerations for Edge‑Based SEO APIs

Open‑SEO optimizes performance by hard‑limiting GSC API rows to 1,000 per call, paginating with startRow, caching expensive payloads in Cloudflare R2/KV, and keeping DB queries minimal—all within Cloudflare Workers' strict CPU‑time constraints.

Open‑SEO is a Cloudflare‑Workers‑based SaaS that pulls data from Google Search Console (GSC) and third‑party APIs, storing results in Cloudflare KV, R2, and D1/Postgres. Because the service runs on edge workers, every request must respect strict limits imposed by the GSC API, the Cloudflare runtime, and underlying storage layers. Understanding these performance considerations ensures your SEO data pipelines remain fast and reliable.

API Row Limits and Pagination Strategy

Google Search Console caps the rows returned per API call. Open‑SEO enforces this constraint at the code level to prevent failures and wasted requests.

In src/server/features/gsc/searchAnalytics.ts, the following constants define the boundaries:

GSC_DEFAULT_ROW_LIMIT = 1000
GSC_MAX_ROW_LIMIT = 1000

When a user requests more data than a single call allows, the system falls back to pagination via startRow. This pattern appears throughout the search performance functions, ensuring large datasets are fetched in bounded chunks rather than attempted in a single, failing request.

Table Pagination Without Extra Count Queries

The UI displays "queries" and "pages" tables with client‑side pagination. Rather than executing a separate COUNT query to determine hasNextPage, the server function fetches pageSize + 1 rows and trims the extra row before returning the response.

This optimization lives in src/serverFunctions/searchPerformance.ts:

// Fetch one extra row to detect next page existence
rowLimit: pageSize + 1,
startRow: (page - 1) * pageSize,

The approach eliminates a database round‑trip while preserving accurate pagination metadata for the frontend.

Striking‑Distance Scan Limits

The "Search Performance" overview computes "striking‑distance" rows—queries ranking just below the first page. To prevent a costly full‑property scan, the code deliberately pulls a limited number of rows (currently 1,000) for this calculation.

This bounded scan, implemented in src/serverFunctions/searchPerformance.ts, trades exhaustive analysis for predictable, sub‑second response times.

Export Row Caps for GSC Compliance

CSV and Google Sheets exports must also respect the GSC per‑call maximum. The EXPORT_ROW_LIMIT constant enforces this boundary:

EXPORT_ROW_LIMIT = 1000

When users request larger exports, the system either paginates internally or surfaces a clear limitation—preventing hung requests and API quota exhaustion.

R2 Caching for Expensive Payloads

Certain operations generate expensive payloads: Lighthouse audits, AI‑search prompts, and large third‑party API responses. Open‑SEO caches these in Cloudflare R2 with a soft TTL stored in object metadata.

The implementation in src/server/lib/r2-cache.ts provides:

// Cheap "hot read" from edge cache
const cached = await getJsonFromR2(cacheKey);
if (cached) return cached;

// On miss: fetch, process, then cache
await putTextToR2(cacheKey, payload, { ttlDays: 30 });

Subsequent reads bypass the origin entirely, reducing latency and eliminating redundant computation.

KV Caching for Short‑Lived Reference Data

Per‑country location lookups and other small, stable datasets live in Cloudflare KV with a 30‑day TTL. This pattern appears in src/server/lib/dataforseo/serp-locations.ts:

const locations = await kv.get("serp-locations:us", "json");
if (!locations) {
  const fresh = await fetchDataForSeoLocations();
  await kv.put("serp-locations:us", JSON.stringify(fresh), {
    expirationTtl: 60 * 60 * 24 * 30,
  });
}

KV's edge‑replicated design makes these lookups effectively free compared to repeated external API calls.

Database Schema Minimization

Open‑SEO keeps D1/Postgres tables lean. Read‑only analytics are served primarily from cache layers (R2, KV), while the relational database holds:

• Project metadata
• User configuration
• Generated reports and user‑created content

This architectural choice, reflected in drizzle.config.ts schema definitions, minimizes database load and query latency.

Edge Runtime Constraints and Background Tasking

Cloudflare Workers enforce a ~50ms CPU‑time limit per request. Open‑SEO respects this by splitting heavy work across boundaries:

Workload Type	Execution Strategy
Real‑time API responses	Lightweight, bounded GSC calls with caching
Large data processing	Background tasks via src/server/mcp/tools/*
Scheduled maintenance	Cron triggers and deferred execution

The search-console-tools.ts and related MCP tools handle bulk operations outside the critical request path, ensuring user‑facing endpoints remain responsive.

Graceful Degradation on Auth Failures

When GSC tokens expire or are revoked, the system detects isExpectedGrantFailure and returns a lightweight "connected": false payload rather than propagating an error. This prevents cascading failures and keeps the UI functional for disconnected properties.

Found in src/server/features/gsc/services/GscService.ts:

if (isExpectedGrantFailure(error)) {
  return { connected: false, error: "token_revoked" };
}

Code Examples

Paginated Performance Table Fetch

// Client request for page 2, 25 rows
const result = await getSearchPerformanceTable({
  projectId: "proj_123",
  dateRange: "last_28_days",
  dimension: "query",
  page: 2,
  pageSize: 25,
});

Server implementation trims the extra detection row automatically.

Capped CSV Export

const csv = await exportSearchPerformanceTable({
  projectId: "proj_123",
  dateRange: "last_7_days",
  dimension: "page",
  // Implicitly capped at EXPORT_ROW_LIMIT (1000)
});

Manual R2 Cache Interaction

import { getJsonFromR2, putTextToR2 } from "@/server/lib/r2";

const cacheKey = "lighthouse/example.com/home";

// Attempt cheap read
const cached = await getJsonFromR2(cacheKey);
if (cached) return cached;

// Populate on miss
const fresh = await runLighthouseAudit(url);
await putTextToR2(cacheKey, JSON.stringify(fresh), {
  metadata: { ttl: Date.now() + 86400000 },
});
return fresh;

Key Source Files

File	Responsibility
src/serverFunctions/searchPerformance.ts	Server functions for UI; enforces limits, pagination, caching
src/server/features/gsc/services/GscService.ts	GSC client wrapper; token handling, error classification
src/server/features/gsc/searchAnalytics.ts	Request schema, limits, date range utilities
src/server/lib/r2-cache.ts	JSON cache layer with soft TTL on R2
src/server/lib/r2.ts	Raw R2 get/put operations
src/server/lib/gscClient.ts	Low‑level GSC HTTP client, error types
drizzle.config.ts	Database schema definitions

Summary

• Hard limits: GSC calls capped at 1,000 rows via GSC_MAX_ROW_LIMIT
• Smart pagination: pageSize + 1 fetches eliminate count queries
• Multi‑layer caching: KV for small data, R2 for large payloads
• Minimal database: Analytics served from cache; DB holds metadata only
• Edge‑aware execution: Heavy work deferred to background tools
• Resilient degradation: Auth failures return lightweight status, not errors

Frequently Asked Questions

What happens if I request more than 1,000 rows from GSC?

Open‑SEO automatically paginates using the startRow parameter. The system fetches data in 1,000‑row chunks and aggregates results internally, transparent to the caller.

Why does Open‑SEO use R2 instead of KV for Lighthouse results?

R2 stores objects up to 5 TB with no per‑key size limits, while KV caps values at 25 MB. Lighthouse JSON and AI prompt responses often exceed KV limits, making R2 the appropriate tier for large, infrequently changing payloads.

How does the edge runtime limit affect my data exports?

Exports complete within the request timeout because they're capped at EXPORT_ROW_LIMIT (1,000) rows. Larger exports require scheduled background tasks rather than synchronous API calls.

What triggers the "connected: false" response?

This occurs when GscService detects an expected grant failure—typically an expired or revoked OAuth token. The UI receives a minimal status payload without the full error stack, allowing graceful re‑authentication flows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}