How OpenSEO Manages the Site Audit Lifecycle: Complete Technical Guide

OpenSEO orchestrates site audits through a state-machine lifecycle spanning validation, reservation, workflow execution, progress tracking, result aggregation, and cleanup, with tier-based capacity enforcement and self-healing capabilities.

OpenSEO implements a robust server-side architecture for managing SEO site audits from initiation to completion. This guide examines how the every-app/open-seo repository handles audit creation, resource limits, distributed execution, and data retention—providing developers with a clear map of the underlying machinery.

Audit Lifecycle Overview

The audit lifecycle in OpenSEO follows nine distinct stages, each implemented by coordinated modules across the codebase. The database row serves as the canonical source of truth, while Cloudflare Workers workflows and KV storage handle operational side effects.

Stage	Primary File	Key Function
Request validation	src/serverFunctions/audit.ts	startAudit
Limit enforcement	src/server/features/audit/services/AuditService.ts	resolveAuditLimitTier
Audit reservation	src/server/features/audit/services/AuditService.ts	startAudit
Concurrency guard	src/server/features/audit/services/AuditService.ts	getAuditUsageForUser
Workflow instantiation	src/server/features/audit/services/AuditService.ts	Workflow creation (L102-116)
Progress tracking	src/server/lib/audit/progress-kv.ts	AuditProgressKV
Status reconciliation	src/server/features/audit/services/AuditService.ts	reconcileRunningAudit
Result aggregation	src/server/features/audit/services/AuditService.ts	getAuditResultsForProject
Cleanup & deletion	src/server/features/audit/services/AuditService.ts	remove

Request Validation and Tier-Based Limits

Every audit begins at the startAudit server function in src/serverFunctions/audit.ts. Input validation uses startAuditSchema from src/types/schemas/audit.ts to enforce structure before any resources are allocated.

Subscription Tier Resolution

The AuditService.resolveAuditLimitTier method (lines 29-44 in src/server/features/audit/services/AuditService.ts) determines capacity based on three tiers:

• Self-hosted: Bypasses all limits
• Free: Enforces strict caps defined in src/shared/audit-limits.ts
• Paid: Elevated capacity per subscription level

// src/shared/audit-limits.ts defines tier boundaries
export const AUDIT_LIMITS = {
  free: { maxPages: 500, maxLighthouseRuns: 10, concurrentAudits: 1 },
  paid: { maxPages: 10000, maxLighthouseRuns: 100, concurrentAudits: 3 },
  selfHosted: null, // unlimited
} as const;

Audit Reservation and Capacity Enforcement

Once validated, AuditService.startAudit calculates resource requirements via getEstimatedAuditCapacity and generates a UUID. The audit row is inserted through AuditRepository.createAudit (lines 76-85).

Before workflow creation, the system performs dual capacity checks:

1. Per-user concurrency: AuditRepository.getAuditUsageForUser counts running audits
2. Overall capacity: Reserved pages and Lighthouse runs against tier limits

Exceeding either limit triggers an AppError with codes AUDIT_ALREADY_RUNNING or AUDIT_CAPACITY_REACHED, aborting the operation before workflow instantiation.

Distributed Workflow Execution

The actual crawl and analysis runs as a Cloudflare Workers workflow instantiated at lines 102-116 in AuditService.ts:

// From src/server/features/audit/services/AuditService.ts#L102-116
const workflow = await env.SITE_AUDIT_WORKFLOW.create({
  id: auditId,
  params: {
    auditId,
    projectId,
    startUrl,
    maxPages,
    lighthouseStrategy,
    // ...additional configuration
  },
});

The workflow executes phases defined in src/server/workflows/siteAuditWorkflowPhases.ts:

1. URL discovery – Crawl start URL and extract links
2. Page crawling – Fetch and analyze HTML for each URL
3. Lighthouse runs – Performance audits per selected strategy
4. Issue detection – Aggregate SEO, accessibility, and best-practice violations

Real-Time Progress and Status Tracking

During execution, intermediate state persists in KV storage via AuditProgressKV. Clients poll progress through getCrawlProgress in src/serverFunctions/audit.ts (lines 71-75), which delegates to AuditProgressKV.getCrawledUrls (lines 12-19 in AuditService.ts).

// Client polling pattern
const pollProgress = async (projectId: string, auditId: string) => {
  const response = await fetch("/api/audit/crawl-progress", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ projectId, auditId }),
  });
  return response.json(); // { crawledUrls: string[], totalDiscovered: number }
};

Self-Healing Status Reconciliation

The getAuditStatus endpoint (lines 50-55 in src/serverFunctions/audit.ts) invokes AuditService.getStatus. A critical reliability feature, reconcileRunningAudit (lines 37-46), detects orphaned audits—those marked running in the database but whose workflow has terminated unexpectedly.

When found, the system attempts state healing to prevent "stuck" audits from indefinitely consuming capacity slots.

Result Aggregation and History

Upon workflow completion, getAuditResults assembles the full audit dataset via AuditRepository.getAuditResultsForProject (lines 64-69). This includes:

• Original audit configuration (parsed via parseAuditConfig)
• Crawled pages with metadata
• Lighthouse performance scores and metrics
• Detected issues categorized by severity

Historical access is lightweight: getAuditHistory (lines 64-69 in src/serverFunctions/audit.ts) returns past audits enriched with a derived ranLighthouse boolean (lines 95-100 in AuditService.ts) to indicate audit depth without loading full results.

Deletion and Resource Cleanup

The deleteAudit flow in AuditService.remove (lines 61-69) ensures complete resource liberation:

1. Running workflow termination: If active, env.SITE_AUDIT_WORKFLOW.get(auditId).terminate() halts execution
2. Database removal: AuditRepository.deleteAuditForProject drops the audit row and associated data
3. Scratchpad destruction: The AuditScratchpad Durable Object—temporary storage during crawls—is explicitly destroyed via getAuditScratchpad(auditId).destroy()

This three-phase cleanup prevents storage leaks and capacity ghosting.

Complete API Usage Example

// 1. Initiate audit creation
const startResponse = await fetch("/api/audit/start", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    projectId: "proj_123",
    startUrl: "https://example.com",
    maxPages: 100,
    lighthouseStrategy: "auto", // "auto", "desktop", "mobile", or "none"
  }),
});
const { auditId } = await startResponse.json();

// 2. Poll until completion
let status;
do {
  await new Promise(r => setTimeout(r, 2000));
  const statusRes = await fetch("/api/audit/status", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ projectId: "proj_123", auditId }),
  });
  status = await statusRes.json();
} while (status.state === "running");

// 3. Fetch comprehensive results
const results = await fetch("/api/audit/results", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ projectId: "proj_123", auditId }),
}).then(r => r.json());

// 4. Export and cleanup
await fetch("/api/audit/delete", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ projectId: "proj_123", auditId }),
});

Key Architectural Components

File	Responsibility
src/serverFunctions/audit.ts	HTTP entry points: startAudit, getAuditStatus, getCrawlProgress, getAuditResults, getAuditHistory, deleteAudit
src/server/features/audit/services/AuditService.ts	Core orchestration: tier resolution, reservation, workflow management, reconciliation, aggregation, removal
src/server/workflows/siteAuditWorkflowPhases.ts	Phase definitions for crawl, Lighthouse, and analysis execution
src/shared/audit-limits.ts	Centralized tier capacity constants
src/types/schemas/audit.ts	Zod schemas for runtime validation
src/server/features/audit/repositories/AuditRepository.ts	Database persistence for audits, pages, Lighthouse data, and issues
src/server/features/audit/AuditScratchpad.ts	Durable Object for temporary crawl data
src/server/lib/audit/progress-kv.ts	KV-backed progress tracking for real-time updates

Summary

• OpenSEO's audit lifecycle is a database-centric state machine with workflow orchestration
• Tier-based enforcement in resolveAuditLimitTier ensures fair resource allocation across free, paid, and self-hosted deployments
• Dual capacity guards prevent overallocation at both user and system levels before workflow creation
• Cloudflare Workers workflows execute distributed crawls through defined phases in siteAuditWorkflowPhases.ts
• Self-healing reconciliation detects and repairs stuck audits to maintain capacity accuracy
• Complete cleanup terminates workflows, removes database records, and destroys scratchpad storage on deletion

Frequently Asked Questions

How does OpenSEO prevent users from exceeding their audit limits?

OpenSEO enforces limits at two points: resolveAuditLimitTier determines the user's subscription cap from src/shared/audit-limits.ts, then getAuditUsageForUser checks current consumption before workflow creation. If either check fails, an AppError aborts the request with specific codes (AUDIT_ALREADY_RUNNING or AUDIT_CAPACITY_REACHED).

What happens if a workflow crashes mid-audit?

The reconcileRunningAudit method in AuditService.ts detects audits marked "running" in the database whose workflow instances no longer exist. It attempts automatic state healing to mark these audits appropriately and free their consumed capacity slots, preventing permanent capacity lockout.

How does self-hosting affect audit lifecycle management?

Self-hosted deployments bypass all limit enforcement when resolveAuditLimitTier detects the self-hosted configuration. The lifecycle proceeds identically through validation, workflow creation, and cleanup—just without capacity gates—making audit-limits.ts values irrelevant for these installations.

Where is crawl progress stored during an active audit?

Progress data streams to KV storage via AuditProgressKV in src/server/lib/audit/progress-kv.ts, while temporary incremental data resides in the AuditScratchpad Durable Object. The KV layer serves real-time polling; the scratchpad supports workflow-internal coordination and is destroyed on audit completion or deletion.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}