# Performance Implications of Running Complex Audits in Open-SEO: A Technical Deep Dive

> Discover the performance implications of complex audits in Open-SEO. Learn about resource consumption, latency, and SQL query scaling for every-app/open-seo.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: deep-dive
- Published: 2026-08-21

---

**Running complex audits in Open-SEO increases resource consumption proportionally with page count, adds 30–90 seconds of latency per Lighthouse batch, and scales SQL query complexity with the size of the persisted dataset, though built-in caps and checkpointing prevent runaway resource usage.**

Open-SEO is an open-source site audit engine that orchestrates multi-phase workflows across Cloudflare Workers. Understanding the performance implications of running complex audits—those configured with high page limits, enabled Lighthouse checks, or extensive issue detection—helps developers optimize throughput, predict execution time, and avoid hitting Worker CPU quotas.

## How Open-SEO Structures Audit Workflows

The audit pipeline is divided into distinct phases, each with specific computational costs. Knowing how these phases interact clarifies where bottlenecks emerge when scaling audit complexity.

### Discovery Phase

The `runDiscoveryPhase` function in [`src/server/workflows/siteAuditWorkflowPhases.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/siteAuditWorkflowPhases.ts) initiates the workflow by pulling URLs from the start page’s sitemap, [`robots.txt`](https://github.com/every-app/open-seo/blob/main/robots.txt), and other entry points. Performance costs here scale with the number of URLs discovered relative to the `maxPages` limit and the parsing overhead of [`robots.txt`](https://github.com/every-app/open-seo/blob/main/robots.txt) rules.

### Crawl Phase

Implemented in [`src/server/workflows/siteAuditWorkflowCrawl.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/siteAuditWorkflowCrawl.ts), the crawl phase visits each discovered URL, stores the raw response, and updates the frontier. This phase generates significant **HTTP request latency** for every page fetched, sustained **database writes** via `AuditRepository.insertPage`, and maintains frontier state in a Durable Object (DO)-based scratchpad. Each URL adds linear overhead in network I/O and disk persistence.

### Lighthouse Phase (Optional)

The `runLighthousePhase` function calls external DataForSEO endpoints for a sampled subset of pages. When `lighthouseStrategy` is set to `"auto"`, the system samples up to **10 pages** and runs both mobile and desktop checks, resulting in **20 external API round-trips** per audit. Each batch introduces 30–90 seconds of latency and persists results via `AuditRepository.insertLighthouseResults`, adding measurable overhead to the total execution window.

### Multipage Checks Phase

Located in [`src/server/lib/audit/issues/multipage.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/issues/multipage.ts), this phase executes SQL-based issue detectors—including broken-link and orphan-page analysis—against the `audit_pages` table. Query complexity increases with the number of persisted pages, potentially forcing the SQL planner into slower join strategies when analyzing link graphs across thousands of rows.

### Finalization Phase

The `finalizeAudit` function aggregates detected issues, records audit completion via `AuditRepository.completeAudit`, and cleans up temporary scratchpad state. Very large audits may generate substantial issue datasets, increasing the cost of final aggregation and potential CSV export operations.

## Critical Performance Cost Drivers

Several configuration parameters directly determine the resource intensity of an audit.

### Page Budget and `maxPages` Configuration

The `AuditConfig.maxPages` parameter defaults to **50** and is enforced by `clampAuditMaxPages` in [`src/server/features/audit/services/audit-capacity.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/audit/services/audit-capacity.ts). Increasing this value to thousands of pages linearly scales **HTTP fetches**, **database row creation**, and **frontier memory storage**. A 10,000-page audit performs approximately 10,000 sequential HTTP requests plus corresponding database insertions, easily consuming multiple minutes of Worker CPU time and saturating I/O quotas.

### Lighthouse Sampling Strategy

The `lighthouseStrategy` configuration determines external API load. The `"auto"` setting samples 10 pages and runs dual-device checks, while `"none"` removes this overhead entirely. Disabling Lighthouse eliminates the 30–90 second per-batch latency and avoids consumption of external API credits.

### SQL-Based Issue Detection

Multipage checks execute pure SQL over the `pages` table. While the codebase mitigates risk by limiting the crawl frontier to `SEED_RPC_BATCH = 2000` and checkpointing intermediate results, audits approaching the upper `maxPages` limit can still push the database query planner into expensive join operations, particularly for link-graph analyses.

### Checkpointing and Memory Pressure

The workflow uses Cloudflare Workers’ `pgStep` checkpointing to survive failures across long-running audits. Each checkpoint adds minor serialization overhead but safeguards progress. The audit scratchpad—a DO stored in Cloudflare KV—maintains the frontier, link graph, and temporary metadata. Its size grows with discovered URL counts, and very large audits may approach DO storage limits, degrading read/write performance.

## Built-In Safeguards Against Resource Exhaustion

Open-SEO implements hard limits to prevent complex audits from exhausting platform resources.

### Per-Tier Audit Limits (`AUDIT_LIMITS`)

The [`src/shared/audit-limits.ts`](https://github.com/every-app/open-seo/blob/main/src/shared/audit-limits.ts) file defines hard caps for free, paid, and self-hosted tiers. These limits constrain the maximum `maxPages` value regardless of user input, ensuring audits remain within predictable resource bounds for the hosting tier’s infrastructure.

### Capacity Clamping (`clampAuditMaxPages`)

Before execution, `AuditService` in [`src/server/features/audit/services/AuditService.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/audit/services/AuditService.ts) passes the requested `maxPages` through `clampAuditMaxPages` in [`src/server/features/audit/services/audit-capacity.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/audit/services/audit-capacity.ts). This function enforces tier-specific boundaries, preventing accidental configuration of resource-prohibitive audits that could trigger Worker timeouts or memory limits.

## Code Examples: Configuring Audit Complexity

Below are practical implementations showing how to adjust audit complexity and the corresponding service-layer handling.

Launch a lightweight audit with default constraints:

```typescript
// 1️⃣ Simple audit: 50 pages max, no Lighthouse
await fetch("/api/audit", {
  method: "POST",
  body: JSON.stringify({
    projectId: "proj_123",
    startUrl: "https://example.com",
  }),
});

```

Launch a resource-intensive audit with expanded scope:

```typescript
// 2️⃣ Complex audit: 200 pages with Lighthouse sampling
await fetch("/api/audit", {
  method: "POST",
  body: JSON.stringify({
    projectId: "proj_123",
    startUrl: "https://example.com",
    maxPages: 200,                 // Increases crawl workload linearly
    lighthouseStrategy: "auto",   // Adds up to 20 external Lighthouse checks
  }),
});

```

Server-side enforcement of capacity limits:

```typescript
// 3️⃣ Inside src/serverFunctions/audit.ts, AuditService respects limits
import { AuditService } from "@/server/features/audit/services/AuditService";

await AuditService.start({
  projectId,
  startUrl,
  maxPages: data.maxPages,               // Automatically clamped by clampAuditMaxPages()
  lighthouseStrategy: data.lighthouseStrategy ?? "auto",
});

```

## Summary

- **Page count linearly scales resource usage**: Higher `maxPages` values in [`src/server/workflows/siteAuditWorkflowCrawl.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/siteAuditWorkflowCrawl.ts) directly increase HTTP latency, database writes, and scratchpad memory consumption.
- **Lighthouse adds significant external latency**: Enabling `"auto"` sampling triggers up to 20 DataForSEO API calls per audit, adding 30–90 seconds of blocking latency.
- **SQL checks worsen with dataset size**: Multipage issue detection in [`src/server/lib/audit/issues/multipage.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/issues/multipage.ts) executes complex joins that slow as the `audit_pages` table grows.
- **Hard caps prevent runaway execution**: The `AUDIT_LIMITS` constant in [`src/shared/audit-limits.ts`](https://github.com/every-app/open-seo/blob/main/src/shared/audit-limits.ts) and the `clampAuditMaxPages` function enforce maximum boundaries appropriate to the hosting tier.

## Frequently Asked Questions

### How does increasing maxPages affect Open-SEO audit performance?

Increasing `maxPages` linearly scales the number of HTTP requests, database insertions via `AuditRepository.insertPage`, and frontier state stored in the DO scratchpad. According to [`src/server/workflows/siteAuditWorkflowCrawl.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/siteAuditWorkflowCrawl.ts), each additional page requires a network fetch and persistence operation, meaning a 10,000-page audit performs roughly 10,000 times the work of a single-page audit, easily exhausting Worker CPU quotas without proper checkpointing.

### What is the latency cost of enabling Lighthouse in complex audits?

Enabling the `"auto"` `lighthouseStrategy` triggers `runLighthousePhase` to sample up to 10 pages and run both mobile and desktop checks against DataForSEO’s API. This generates 20 external round-trips that typically add 30–90 seconds of latency per audit batch. Disabling Lighthouse by setting the strategy to `"none"` removes this overhead entirely, as implemented in [`src/server/workflows/siteAuditWorkflowPhases.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/siteAuditWorkflowPhases.ts).

### How does Open-SEO prevent audits from consuming excessive resources?

The platform enforces tier-specific limits through `AUDIT_LIMITS` in [`src/shared/audit-limits.ts`](https://github.com/every-app/open-seo/blob/main/src/shared/audit-limits.ts) and clamps user input via `clampAuditMaxPages` in [`src/server/features/audit/services/audit-capacity.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/audit/services/audit-capacity.ts). These safeguards run before the crawl phase begins, ensuring requested page counts never exceed infrastructure capacity regardless of API input.

### Where are multipage SQL checks executed in the codebase?

Multipage issue detection occurs in [`src/server/lib/audit/issues/multipage.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/issues/multipage.ts) within the `runMultipageChecks` function. This module runs SQL queries against the `audit_pages` table to detect issues like broken links and orphan pages. Query performance degrades with table size because link-graph analyses require expensive joins across the entire persisted dataset.