How Lighthouse Scoring Integrates Into the Site Audit Workflow in open-seo

Lighthouse scoring operates as a conditional, post-crawl phase within the open-seo site-audit workflow that samples representative pages, executes parallel mobile and desktop analysis via the DataForSEO API, stores raw JSON payloads in Cloudflare R2, and persists normalized scores to the audit database for downstream reporting.

The open-seo repository implements a phased workflow architecture for comprehensive site auditing. Lighthouse scoring represents a critical evaluation stage that executes only after the crawl phase completes, enabling automated performance and SEO quality assessment across a curated subset of discovered pages. This integration leverages external API infrastructure while maintaining efficient cloud storage and real-time progress tracking mechanisms.

Phase Orchestration and Conditional Execution

The workflow engine initiates Lighthouse analysis through the runAuditPhases function in src/server/workflows/siteAuditWorkflowPhases.ts. This orchestrator invokes runLighthousePhase immediately after the crawl step succeeds, passing the audit context including auditId, projectId, and configuration parameters.

await runLighthousePhase(step, {
  auditId,
  workflowInstanceId,
  billingCustomer,
  projectId,
  startUrl,
  config,
});

The phase executes conditionally based on the config.lighthouseStrategy value. If this configuration is set to "none", the entire Lighthouse phase is bypassed (lines 89‑90), allowing audits to run without performance analysis when unnecessary.

Intelligent Page Sampling Strategy

Before fetching metrics, the system selects a representative page sample to minimize API costs while maintaining coverage. The selectLighthousePages function queries the database for all crawled pages, then delegates sampling logic to selectLighthouseSample in src/server/lib/audit/lighthouse.ts.

The sampling algorithm follows these rules:

• Always include the start URL to ensure homepage analysis
• Filter for successful responses only (HTTP 2xx status codes)
• Apply template diversity: For the "auto" strategy, select up to 10 pages—one per distinct URL template—to capture varied page architectures
• Return structured objects containing {url, pageId} pairs for the subsequent fetch step

This approach ensures that single-page applications and complex multi-template sites receive appropriate coverage without exhausting API quotas on redundant page types.

Parallel Lighthouse Fetching via DataForSEO API

For each sampled URL, the workflow creates processing chunks of concurrent size 5 and executes two parallel fetches—mobile and desktop—via fetchLighthouseResult. This function in src/server/lib/audit/lighthouse.ts instantiates a DataForSEO client and invokes dataforseo.lighthouse.live() with the target URL and device strategy.

The fetcher extracts critical scoring fields and metrics from the API response:

const data = await dataforseo.lighthouse.live({ url, strategy });
return {
  result: {
    url,
    pageId,
    strategy,
    performanceScore: data.scores.performance,
    accessibilityScore: data.scores.accessibility,
    bestPracticesScore: data.scores['best-practices'],
    seoScore: data.scores.seo,
    lcpMs: data.metrics.lcpMs,
    cls: data.metrics.cls,
    inpMs: data.metrics.inpMs,
    ttfbMs: data.metrics.ttfbMs,
  },
  payloadJson: JSON.stringify(data),
};

Error handling ensures workflow resilience: failed requests generate failedLighthouseFetch records containing null scores and the specific error message, allowing the audit to continue while logging failures for review.

Persistent Storage Architecture

After fetching, storeLighthouseResult handles dual persistence requirements. First, it writes the complete raw JSON payload to Cloudflare R2 using the path pattern:

site-audit/<projectId>/<auditId>/<pageId>-<strategy>.json

This storage strategy preserves full Lighthouse reports for deep analysis while keeping the operational database lean. The function returns a LighthouseResult object containing the extracted scores plus metadata including r2Key and payloadSizeBytes.

Subsequently, AuditRepository.insertLighthouseResults persists the normalized score data to the relational database (defined in src/db/audit.schema.ts), enabling fast querying for UI dashboards and reporting APIs.

Real-Time Progress Tracking and Finalization

Throughout the Lighthouse phase, the workflow maintains accurate execution state through AuditRepository.updateAuditProgress. The system tracks:

• lighthouseTotal: Total number of URL/strategy combinations to process
• lighthouseCompleted: Successfully analyzed pages
• lighthouseFailed: Requests that encountered errors

These counters surface in the user interface to provide visibility into audit completion status.

After Lighthouse processing concludes, the finalizeAudit function executes multipage validation checks and records the boolean flag run_lighthouse (set to true when config.lighthouseStrategy !== "none"). This marker allows downstream consumers to distinguish between audits that included performance analysis versus crawl-only executions.

Summary

• Lighthouse scoring executes as a dedicated phase within the open-seo workflow, triggered only after successful crawl completion and contingent on configuration settings.
• Page sampling intelligently selects up to 10 representative URLs plus the start page, grouped by URL template to ensure architectural diversity.
• Concurrent processing handles mobile and desktop analysis in parallel chunks of 5 requests, maximizing throughput while respecting API rate limits.
• Dual storage separates concerns: raw JSON payloads reside in Cloudflare R2 for archival purposes, while normalized scores populate the audit database for operational queries.
• Resilient error handling converts fetch failures into null-score records, ensuring workflow completion even when individual pages fail analysis.

Frequently Asked Questions

What determines whether the Lighthouse phase runs in a site audit?

The config.lighthouseStrategy parameter controls phase execution. When set to "none", the workflow skips Lighthouse entirely and proceeds directly to finalization. Any other value (such as "auto") triggers the sampling and analysis sequence, with the flag ultimately recorded in the database as run_lighthouse.

How does open-seo decide which pages to analyze with Lighthouse?

The system queries all successfully crawled pages (HTTP 2xx responses) and applies the selectLighthouseSample algorithm. This function always includes the startUrl, then groups remaining pages by URL template pattern and selects one representative per group, capping the sample at 10 pages to balance coverage with API consumption.

Where are the complete Lighthouse reports stored in the open-seo architecture?

Raw JSON payloads containing full Lighthouse results are stored in Cloudflare R2 under the key pattern site-audit/<projectId>/<auditId>/<pageId>-<strategy>.json. Normalized score data—including performance, accessibility, best-practices, and SEO scores—are persisted to the relational database via AuditRepository.insertLighthouseResults for dashboard rendering.

What happens when a Lighthouse API request fails during the audit workflow?

Failed requests are captured by error handling logic in fetchLighthouseResult, which generates a failedLighthouseFetch record containing null scores and the specific error message. The workflow increments the lighthouseFailed counter and continues processing remaining chunks, ensuring partial audit completion rather than total workflow failure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}