How OpenSEO Integrates Lighthouse Data Into Site Audits

OpenSEO enriches site audits by sampling up to 10 representative pages post-crawl, executing Google Lighthouse via the DataForSEO API for both mobile and desktop strategies, and storing raw JSON payloads in Cloudflare R2 while persisting summaries to the D1 database.

OpenSEO’s server-side architecture incorporates Google Lighthouse metrics through a dedicated workflow phase that follows the standard crawling process. After the discovery and crawling phases map your site structure, the system triggers a targeted Lighthouse analysis to capture granular performance, accessibility, and best-practice diagnostics. This Lighthouse data integration ensures comprehensive audit reports combine broad site coverage with deep technical metrics derived from real browser environments.

The Three-Phase Audit Workflow

OpenSEO organizes site audits into three sequential phases orchestrated within src/server/workflows/siteAuditWorkflowPhases.ts. The workflow begins with discovery (sitemap/seed URL processing), proceeds to crawling (headless browser page analysis), and concludes with an optional Lighthouse phase for detailed performance metrics.

After the crawler finishes gathering page data, the runAuditPhases function invokes runLighthousePhase (lines 83-92). This transition ensures Lighthouse analysis operates on verified, crawlable URLs rather than theoretical sitemap entries. The Lighthouse phase receives the complete list of crawled pages and initiates the sampling and analysis pipeline.

Intelligent Page Sampling Strategy

To balance computational cost with diagnostic value, OpenSEO analyzes a maximum of 10 pages per audit. The selection logic prioritizes coverage across different URL templates while guaranteeing the start page is always included.

Page Selection Logic

The selectLighthouseSample function in src/server/lib/audit/lighthouse.ts (lines 24-71) implements the sampling algorithm:

• Filters for successful HTTP responses (2xx status codes only)
• Always includes the start URL via canonical key matching
• Groups remaining pages by detected URL templates (e.g., /blog/:slug, /products/:id)
• Selects one representative page per template group until reaching the 10-page limit

// src/server/lib/audit/lighthouse.ts
export function selectLighthouseSample(
  pages: LighthouseSamplePage[],
  startUrl: string,
  strategy: LighthouseStrategy,
): string[] {
  if (strategy === "none") return [];

  const validPages = pages.filter(p => p.statusCode >= 200 && p.statusCode < 300);
  const selected = new Set<string>();

  // always include the start page
  const startKey = canonicalUrlKey(startUrl);
  const startPage =
    validPages.find(p => canonicalUrlKey(p.url) === startKey) ??
    validPages.find(p =>
      canonicalUrlKeyWithoutTrailingSlash(p.url) ===
      canonicalUrlKeyWithoutTrailingSlash(startUrl),
    );
  if (startPage) selected.add(startPage.url);

  // group by URL‑template and pick one per group (max 10)
  const templateGroups = new Map<string, LighthouseSamplePage>();
  for (const page of validPages) {
    if (selected.has(page.url)) continue;
    const template = detectUrlTemplate(new URL(page.url).pathname);
    if (!templateGroups.has(template)) templateGroups.set(template, page);
  }
  for (const [, page] of templateGroups) {
    if (selected.size >= 10) break;
    selected.add(page.url);
  }
  return Array.from(selected);
}

Fetching and Storing Lighthouse Results

For each selected URL, OpenSEO executes dual-strategy analysis to capture performance characteristics across device types. The system fetches results via the DataForSEO API and implements a hybrid storage architecture separating raw payloads from queryable summaries.

Dual-Strategy Execution

Within runLighthousePhase, the workflow calls fetchAndStoreLighthouseResult twice per page—once with strategy: "mobile" and once with strategy: "desktop" (lines 176-194). This ensures audit reports reflect mobile-specific constraints (CPU throttling, viewport emulation) alongside desktop baselines.

Cloudflare R2 Storage Architecture

The fetchAndStoreLighthouseResult function in src/server/lib/audit/lighthouse.ts manages the complete fetch-store lifecycle:

1. API Integration: Calls the DataForSEO "live Lighthouse" endpoint via createDataforseoClient (lines 31-46)
2. Object Storage: Serializes the raw Lighthouse JSON and uploads it to Cloudflare R2 under a namespaced key: site-audit/${projectId}/${auditId}/${pageId}-${strategy}.json (lines 13-21)
3. Metadata Return: Returns a LighthouseResult object containing the R2 key and payload size for later retrieval (lines 94-21)

// src/server/lib/audit/lighthouse.ts
export async function fetchAndStoreLighthouseResult(input: {
  url: string;
  pageId: string;
  strategy: "mobile" | "desktop";
  billingCustomer: BillingCustomerContext;
  projectId: string;
  auditId: string;
}): Promise<LighthouseResult> {
  const fetched = await fetchLighthouseResult(
    input.url,
    input.pageId,
    input.strategy,
    input.billingCustomer,
  );

  if (!fetched.payloadJson) return fetched.result;

  const key = `site-audit/${input.projectId}/${input.auditId}/${input.pageId}-${input.strategy}.json`;
  const uploaded = await putTextToR2(key, fetched.payloadJson);

  return {
    ...fetched.result,
    r2Key: uploaded.key,
    payloadSizeBytes: uploaded.sizeBytes,
  };
}

After both mobile and desktop results are fetched, AuditRepository.insertLighthouseResults persists summary rows to the D1 (SQLite/PostgreSQL) database (lines 99-100), enabling fast querying without downloading full JSON payloads.

Retrieving and Exporting Audit Data

OpenSEO exposes Lighthouse data through type-safe server functions that decode stored payloads and format results for consumption by the React frontend or export utilities.

Server-Side API Endpoints

The src/serverFunctions/lighthouse.ts module defines two primary endpoints:

• getAuditLighthouseIssues: Retrieves decoded scores, metrics, and issue lists by reading the R2 payload via readStoredLighthousePayload (lines 45-68)
• exportAuditLighthouseIssues: Generates downloadable CSV/JSON exports via buildLighthouseExportFile (lines 69-88)

// src/serverFunctions/lighthouse.ts
export const getAuditLighthouseIssues = createServerFn({ method: "POST" })
  .middleware(requireProjectContext)
  .validator(lighthouseAuditIssueSchema)
  .handler(async ({ data, context }) => {
    const lighthouse = await getAuditLighthouseData({
      projectId: context.projectId,
      resultId: data.resultId,
    });

    return {
      id: lighthouse.id,
      finalUrl:
        lighthouse.payload.storedPayload?.metadata.finalUrl ??
        lighthouse.finalUrl,
      strategy: lighthouse.strategy,
      createdAt: lighthouse.createdAt,
      hasIssueDetails: lighthouse.payload.report.hasIssueDetails,
      scores: lighthouse.payload.storedPayload?.scores ?? null,
      metrics: lighthouse.payload.storedPayload?.metrics ?? null,
      issues: lighthouse.payload.report.issues,
    };
  });

Payload Decoding and Validation

Raw JSON stored in R2 is decoded by src/server/lib/lighthousePayload.ts, which validates the schema against TypeScript definitions in src/shared/lighthouse.ts, extracts performance metrics, accessibility scores, and category-specific issues, and applies optional filters before returning the structured report.

Summary

• Three-phase architecture: Lighthouse analysis runs as a dedicated phase after discovery and crawling, triggered by runLighthousePhase in siteAuditWorkflowPhases.ts
• Intelligent sampling: The selectLighthouseSample function analyzes up to 10 pages (start URL plus one per template) to ensure broad coverage without excessive API costs
• Dual-strategy analysis: Every sampled page runs through both mobile and desktop Lighthouse configurations via fetchAndStoreLighthouseResult
• Hybrid storage model: Raw JSON payloads reside in Cloudflare R2 for archival access, while D1 database rows store queryable summaries for fast dashboard rendering
• Type-safe retrieval: Server functions in serverFunctions/lighthouse.ts provide structured access to scores, metrics, and exportable issue lists

Frequently Asked Questions

How does OpenSEO select which pages to analyze with Lighthouse?

OpenSEO uses the selectLighthouseSample function in src/server/lib/audit/lighthouse.ts to choose up to 10 pages from the complete crawl results. The algorithm always includes the start URL, filters for successful 2xx responses, groups pages by URL template patterns, and selects one representative per group to ensure diverse coverage across site sections.

Where does OpenSEO store the raw Lighthouse JSON data?

Raw Lighthouse JSON payloads are stored in Cloudflare R2 under namespaced keys following the pattern site-audit/${projectId}/${auditId}/${pageId}-${strategy}.json. This object storage approach preserves the complete audit artifacts while keeping the primary D1 database lean with only metadata and summary scores for fast querying.

Can I export Lighthouse results from an OpenSEO audit?

Yes. The exportAuditLighthouseIssues server function in src/serverFunctions/lighthouse.ts generates downloadable export files in CSV or JSON format. This endpoint utilizes buildLighthouseExportFile to compile scores, metrics, and detailed issue lists from the stored R2 payloads into standardized report formats suitable for client delivery or further analysis.

What Lighthouse strategies does OpenSEO support?

OpenSEO supports mobile and desktop strategies. During the audit workflow, the system automatically executes both strategies for every sampled page by calling fetchAndStoreLighthouseResult twice—once per strategy. This dual execution captures device-specific performance characteristics including mobile CPU throttling and viewport emulation settings.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}