How OpenSEO Utilizes Lighthouse for Comprehensive Site Audits

OpenSEO integrates Lighthouse audits by sampling up to 10 representative pages from crawl results, running both mobile and desktop analyses via the DataForSEO API, and storing serialized JSON payloads in Cloudflare R2 for later retrieval and reporting.

The every-app/open-seo repository implements a multi-phase site audit workflow that extends traditional crawling with Google Lighthouse performance diagnostics. This integration enables comprehensive scoring across performance, accessibility, best practices, and SEO categories without requiring local Chrome infrastructure.

The Three-Phase Audit Architecture

OpenSEO’s site audit process follows a structured pipeline consisting of discovery, crawling, and an optional Lighthouse phase. According to the source code in src/server/workflows/siteAuditWorkflowPhases.ts, after the crawler aggregates a complete list of pages, the runAuditPhases function invokes runLighthousePhase to begin the analysis (lines 83–92).

This modular approach ensures Lighthouse runs only after successful crawl completion, receiving the full inventory of discovered pages as input for intelligent sampling.

Selecting Representative Pages for Lighthouse Analysis

Rather than auditing every crawled URL, OpenSEO applies intelligent filtering to balance coverage against API quota and execution time.

Page Selection Logic

The selectLighthouseSample function in src/server/lib/audit/lighthouse.ts (lines 24–71) implements a deduplication strategy that:

• Filters for 2xx HTTP status codes only
• Always includes the start page
• Groups remaining pages by detected URL template
• Selects one page per template group until reaching a maximum of 10 URLs

// src/server/lib/audit/lighthouse.ts
export function selectLighthouseSample(
  pages: LighthouseSamplePage[],
  startUrl: string,
  strategy: LighthouseStrategy,
): string[] {
  if (strategy === "none") return [];

  const validPages = pages.filter(p => p.statusCode >= 200 && p.statusCode < 300);
  const selected = new Set<string>();

  // always include the start page
  const startKey = canonicalUrlKey(startUrl);
  const startPage =
    validPages.find(p => canonicalUrlKey(p.url) === startKey) ??
    validPages.find(p =>
      canonicalUrlKeyWithoutTrailingSlash(p.url) ===
      canonicalUrlKeyWithoutTrailingSlash(startUrl),
    );
  if (startPage) selected.add(startPage.url);

  // group by URL‑template and pick one per group (max 10)
  const templateGroups = new Map<string, LighthouseSamplePage>();
  for (const page of validPages) {
    if (selected.has(page.url)) continue;
    const template = detectUrlTemplate(new URL(page.url).pathname);
    if (!templateGroups.has(template)) templateGroups.set(template, page);
  }
  for (const [, page] of templateGroups) {
    if (selected.size >= 10) break;
    selected.add(page.url);
  }
  return Array.from(selected);
}

This grouping mechanism ensures diverse page types (such as product pages versus blog posts) receive coverage without redundant testing of similar templates.

Fetching and Storing Lighthouse Results

Once URLs are selected, OpenSEO executes parallel audits for both device types and persists the raw data for downstream analysis.

Dual Strategy Execution

For each sampled URL, runLighthousePhase calls fetchAndStoreLighthouseResult twice—once with strategy: "mobile" and once with strategy: "desktop" (as seen in src/server/workflows/siteAuditWorkflowPhases.ts lines 176–194). This dual execution ensures responsive design issues are captured across viewport contexts.

DataForSEO Integration and R2 Storage

The fetchAndStoreLighthouseResult function in src/server/lib/audit/lighthouse.ts (lines 31–46) interfaces with the DataForSEO "live Lighthouse" endpoint via createDataforseoClient. After fetching the raw Lighthouse JSON, it serializes the payload and uploads it to Cloudflare R2 using a namespaced key structure:

// src/server/lib/audit/lighthouse.ts
export async function fetchAndStoreLighthouseResult(input: {
  url: string;
  pageId: string;
  strategy: "mobile" | "desktop";
  billingCustomer: BillingCustomerContext;
  projectId: string;
  auditId: string;
}): Promise<LighthouseResult> {
  const fetched = await fetchLighthouseResult(
    input.url,
    input.pageId,
    input.strategy,
    input.billingCustomer,
  );

  if (!fetched.payloadJson) return fetched.result;

  const key = `site-audit/${input.projectId}/${input.auditId}/${input.pageId}-${input.strategy}.json`;
  const uploaded = await putTextToR2(key, fetched.payloadJson);

  return {
    ...fetched.result,
    r2Key: uploaded.key,
    payloadSizeBytes: uploaded.sizeBytes,
  };
}

The function returns a LighthouseResult object containing the R2 key and payload size, which AuditRepository.insertLighthouseResults then persists to the D1 (SQLite/PostgreSQL) database for indexing (lines 99–100 in the workflow file).

Retrieving and Exporting Audit Data

Stored Lighthouse data is exposed through type-safe server functions that decode raw payloads and transform them into actionable reports.

API Endpoints for Data Access

The server-side API defined in src/serverFunctions/lighthouse.ts provides two primary endpoints:

• getAuditLighthouseIssues – Retrieves stored R2 payloads using readStoredLighthousePayload, returning scores, metrics, and categorized issues (lines 45–68)
• exportAuditLighthouseIssues – Generates downloadable CSV/JSON exports via buildLighthouseExportFile (lines 69–88)

// src/serverFunctions/lighthouse.ts
export const getAuditLighthouseIssues = createServerFn({ method: "POST" })
  .middleware(requireProjectContext)
  .validator(lighthouseAuditIssueSchema)
  .handler(async ({ data, context }) => {
    const lighthouse = await getAuditLighthouseData({
      projectId: context.projectId,
      resultId: data.resultId,
    });

    return {
      id: lighthouse.id,
      finalUrl:
        lighthouse.payload.storedPayload?.metadata.finalUrl ??
        lighthouse.finalUrl,
      strategy: lighthouse.strategy,
      createdAt: lighthouse.createdAt,
      hasIssueDetails: lighthouse.payload.report.hasIssueDetails,
      scores: lighthouse.payload.storedPayload?.scores ?? null,
      metrics: lighthouse.payload.storedPayload?.metrics ?? null,
      issues: lighthouse.payload.report.issues,
    };
  });

Payload Decoding and Issue Extraction

Raw JSON stored in R2 is decoded by src/server/lib/lighthousePayload.ts, which validates the schema against shared types defined in src/shared/lighthouse.ts. This module extracts performance metrics, accessibility scores, and diagnostic audits while filtering by category for targeted reporting.

Summary

• OpenSEO runs Lighthouse as an optional third phase after discovery and crawling, orchestrated by runAuditPhases in the site audit workflow.
• The system intelligently samples up to 10 pages using template-based deduplication via selectLighthouseSample to ensure diverse coverage without excessive API usage.
• Each selected page undergoes dual-device testing (mobile and desktop) through the DataForSEO API, with results stored as serialized JSON in Cloudflare R2.
• Database records maintained by AuditRepository.insertLighthouseResults index the R2 locations, enabling efficient retrieval through getAuditLighthouseIssues and export functionality.

Frequently Asked Questions

How many pages does OpenSEO audit with Lighthouse?

OpenSEO limits Lighthouse analysis to 10 representative pages per site audit. The selection algorithm always includes the start page and one additional page per detected URL template, ensuring coverage of distinct page types while respecting API quotas and processing time constraints.

What Lighthouse strategies does OpenSEO support?

OpenSEO executes both mobile and desktop strategies for every sampled URL. The workflow explicitly fetches results with strategy: "mobile" and strategy: "desktop" parameters, storing both payloads separately in R2 to enable cross-device performance comparisons.

Where does OpenSEO store the raw Lighthouse JSON data?

Raw Lighthouse payloads are stored in Cloudflare R2 under a hierarchical key pattern: site-audit/{projectId}/{auditId}/{pageId}-{strategy}.json. This object storage approach preserves the complete audit data for long-term analysis while the D1 database maintains lightweight metadata records linking to these stored objects.

Can Lighthouse results be exported from OpenSEO?

Yes. The system provides an exportAuditLighthouseIssues server function that generates downloadable CSV or JSON exports containing scores, metrics, and diagnostic issues. This function utilizes buildLighthouseExportFile to format the decoded payload data for external reporting tools.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}