Performance Considerations for Open-SEO: Optimizing Cloudflare Workers, API Limits, and Edge Caching

Open-SEO mitigates Cloudflare Worker CPU constraints and Google Search Console API quotas through strict row caps, strategic pagination, multi-layer caching in KV and R2, and delegation of heavy processing to background tasks.

Open-SEO is a Cloudflare-Workers-based SaaS that aggregates search analytics from Google Search Console (GSC) and third-party APIs. Because the platform executes entirely at the edge under strict runtime limits, understanding the performance considerations for open-seo requires analyzing how the codebase manages API quotas, memory constraints, and storage optimization across KV, R2, and D1 databases.

API Rate Limits and Row Cap Management: Core Performance Considerations for Open-SEO

Google Search Console imposes strict caps on data retrieval, which Open-SEO enforces at the library level to prevent runtime failures.

Enforcing GSC Query Limits

In src/server/features/gsc/searchAnalytics.ts, the codebase defines hard constants that bound every request:

• GSC_DEFAULT_ROW_LIMIT: Set to 1,000 rows
• GSC_MAX_ROW_LIMIT: Capped at 1,000 rows

These limits ensure that the searchAnalytics request never exceeds the GSC API maximum. When users request larger datasets, the system automatically engages pagination via the startRow parameter, slicing the workload into bounded chunks.

Pagination Without Count Queries

The table views in the UI implement an efficient pagination pattern in src/serverFunctions/searchPerformance.ts. Instead of issuing an expensive COUNT query to determine if more pages exist, the server fetches pageSize + 1 rows. If the extra row exists, the function sets hasNextPage: true and trims the result before returning the payload. This eliminates an entire round-trip to the database or API.

Edge Caching and Storage Optimization: Performance Considerations for Open-SEO

Open-SEO minimizes redundant computation by storing expensive API responses and frequently accessed metadata in Cloudflare's edge storage layers.

R2 Object Storage for Heavy Payloads

Expensive operations—such as Lighthouse audits and AI-search prompts—are cached in Cloudflare R2 with a soft TTL mechanism. The src/server/lib/r2-cache.ts module stores JSON payloads with expiration metadata directly in the object headers. Subsequent requests retrieve cached data via cheap "hot reads" from the edge, bypassing origin fetches entirely.

KV Storage for Short-Lived Data

Location lookups and other small, quasi-static blobs are stored in Cloudflare KV with a 30-day TTL. The src/server/lib/dataforseo/serp-locations.ts implementation reduces repeat external fetches for per-country location data, keeping Worker execution time minimal.

Database and Compute Optimization: Performance Considerations for Open-SEO

The architecture keeps the relational database lean while offloading heavy computation from the request path.

Lean Schema Design

The D1 SQLite and Postgres schemas, defined in drizzle.config.ts and related migration files, maintain minimal tables for project metadata and user-generated data. Read-only analytics are served primarily from the cache layers (R2 and KV), ensuring the database handles only transactional writes and lookups.

Background Task Delegation

Cloudflare Workers enforce a CPU time limit of approximately 50ms per request. To comply with this constraint, Open-SEO splits heavy lifting—such as large API calls and batch processing—into background tasks located in src/server/mcp/tools/. These scheduled workers process data outside the critical request path, preventing timeouts for end users.

Graceful Degradation: Resilience as a Performance Consideration for Open-SEO

When external services fail, the system preserves frontend performance by returning lightweight fallback states rather than crashing the request.

Token Failure Detection

In src/server/features/gsc/services/GscService.ts, the isExpectedGrantFailure utility detects missing or revoked GSC tokens. Instead of throwing a full stack trace, the service returns a compact { connected: false } payload. This allows the UI to render a disconnected state instantly without wasting cycles on failed API retries.

Implementation Examples for Open-SEO Performance Patterns

The following patterns demonstrate how to interact with Open-SEO's performance optimizations in your own extensions or API consumers.

Fetching Paginated Performance Data

To request a specific page of query data without triggering expensive count operations:

const result = await getSearchPerformanceTable({
  projectId: "proj_123",
  dateRange: "last_28_days",
  dimension: "query",
  page: 2,
  pageSize: 25,
});

The underlying handler in src/serverFunctions/searchPerformance.ts translates this into a GSC request with rowLimit: 26 and startRow: 25, returning exactly 25 rows plus a boolean indicating if additional pages exist.

Exporting Data Within API Limits

When exporting search performance to CSV or Google Sheets, the system respects the 1,000-row cap:

const csv = await exportSearchPerformanceTable({
  projectId: "proj_123",
  dateRange: "last_7_days",
  dimension: "page",
});

The export function automatically clamps the request to EXPORT_ROW_LIMIT = 1000, ensuring compatibility with GSC constraints while preventing Worker timeouts.

Caching Expensive Computations

To leverage the R2 cache layer for custom heavy computations:

import { getJsonFromR2, putTextToR2 } from "@/server/lib/r2";

const cacheKey = "lighthouse/audit-12345";
let payload = await getJsonFromR2(cacheKey);

if (!payload) {
  payload = await runExpensiveLighthouseAudit();
  await putTextToR2(cacheKey, JSON.stringify(payload));
}

This pattern, utilized throughout src/server/lib/r2-cache.ts, ensures that subsequent requests for the same audit retrieve data from the edge in milliseconds rather than recomputing results.

Summary

Open-SEO optimizes for edge constraints through several coordinated strategies:

• Hard API limits: Enforces 1,000-row maximums in src/server/features/gsc/searchAnalytics.ts with automatic pagination via startRow
• Efficient pagination: Uses pageSize + 1 fetching in src/serverFunctions/searchPerformance.ts to eliminate count queries
• Multi-tier caching: Stores heavy payloads in R2 (with soft TTL metadata) and short-lived data in KV (with 30-day TTL)
• Database minimalism: Keeps D1/Postgres schemas lean, serving analytics from cache rather than relational queries
• Background processing: Offloads heavy computation to MCP tools in src/server/mcp/tools/ to stay within 50ms CPU limits
• Graceful degradation: Detects token failures via isExpectedGrantFailure in src/server/features/gsc/services/GscService.ts to return lightweight disconnected states

Frequently Asked Questions

What are the hard limits for GSC API queries in Open-SEO?

The codebase enforces a GSC_MAX_ROW_LIMIT of 1,000 rows per request in src/server/features/gsc/searchAnalytics.ts. This constant ensures the application never requests more data than the Google Search Console API allows in a single call. For larger datasets, the system automatically paginates using the startRow parameter.

How does Open-SEO handle expensive operations without hitting Worker CPU limits?

Heavy processing is delegated to background tasks located in src/server/mcp/tools/. These tasks run outside the HTTP request lifecycle, allowing the main Workers to return responses within the approximately 50ms CPU time limit. Additionally, expensive computations are cached in R2 with soft TTLs to minimize repeat execution.

What caching layers does Open-SEO use to optimize performance?

The system employs a dual-layer strategy: Cloudflare R2 stores large, expensive payloads like Lighthouse audits with expiration metadata for soft TTL management, while Cloudflare KV handles small, frequently accessed data such as SERP locations with a 30-day TTL. This hierarchy ensures that read-heavy analytics bypass the database entirely.

How does the application behave when GSC tokens expire or are revoked?

The GscService class in src/server/features/gsc/services/GscService.ts detects authentication failures via the isExpectedGrantFailure predicate. Rather than throwing an error that bubbles up to the user, it returns a lightweight JSON payload containing { connected: false }. This allows the frontend to render a disconnected state instantly without consuming additional API quota or Worker CPU time.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}