How OpenSEO Fetches Lighthouse Performance Metrics: Complete Technical Breakdown

OpenSEO retrieves Lighthouse performance metrics through a three-stage pipeline that samples representative URLs, executes live requests to the DataForSEO API, and normalizes responses into typed storage objects using Zod validation.

The open-source SEO platform OpenSEO automates website auditing by integrating Google Lighthouse metrics directly into its analysis workflow. Understanding how the system fetches these Lighthouse performance metrics requires examining the tight coupling between URL sampling logic, external API orchestration, and data normalization layers implemented in TypeScript.

URL Sampling and Audit Orchestration

The fetching process begins within the site audit workflow defined in src/server/workflows/siteAuditWorkflowPhases.ts. During the audit phases, the system invokes selectLighthouseSample() (located at lines 97-124 in src/server/lib/audit/lighthouse.ts) to intelligently choose up to ten representative URLs from the crawled site.

This sampling strategy prioritizes the homepage and selects one page per unique URL pattern, ensuring coverage across different page templates without overwhelming the API with redundant checks. The function accepts a strategy parameter (auto, mobile, or desktop) that determines the device emulation for subsequent Lighthouse runs.

Live DataForSEO API Integration

For each selected URL, the workflow triggers fetchLighthouseResult() (lines 25-55 of src/server/lib/audit/lighthouse.ts), the primary public-facing helper for external data retrieval. This function executes the following sequence:

1. Client instantiation – Creates a DataForSEO client via createDataforseoClient(billingCustomer) imported from src/server/lib/dataforseo.ts.

2. API execution – Calls dataforseo.lighthouse.live({ url, strategy }), which delegates to the low-level request handler in src/server/lib/dataforseo/lighthouse.ts.

3. Payload construction – The low-level function builds a OnPageLighthouseLiveJsonRequestInfo object and POSTs it to DataForSEO's /lighthouse/live/json endpoint (lines 16-26).

The raw JSON response contains unvalidated Lighthouse audit data that requires transformation before storage.

Response Parsing and Validation

Raw API responses flow into parseDataforseoLighthousePayload() defined in src/server/lib/dataforseoLighthousePayload.ts (lines 2-71). This normalization routine performs critical data integrity operations:

• Schema validation – Enforces structure using dataforseoLighthouseResponseSchema and related Zod schemas to reject malformed payloads.
• Metric extraction – Pulls category scores for performance, accessibility, best-practices, and seo, alongside key metrics including largestContentfulPaint, cumulativeLayoutShift, interactionToNextPaint, and serverResponseTime.
• Score normalization – Converts decimal scores to percentages via scoreToPercent and assembles a StoredLighthousePayload containing metadata, scores, metrics, and detailed issue descriptions.

Data Persistence and Storage

Once normalized, the higher-level storeLighthouseResult() function (lines 74-92 in src/server/lib/audit/lighthouse.ts) handles long-term storage. This helper optionally uploads the JSON payload to Cloudflare R2 using utilities from src/server/lib/r2.ts, returning a LighthouseResult object enriched with storage keys and payload size metadata.

The site audit workflow then merges these enriched results into the comprehensive audit report, updating progress counters to reflect completion status.

Implementation Example

The following examples demonstrate manual invocation of the Lighthouse fetching utilities:

// Fetching Lighthouse data for a specific URL
import { fetchLighthouseResult } from "@/server/lib/audit/lighthouse";
import { createDataforseoClient } from "@/server/lib/dataforseo";

async function getMetrics() {
  const billingCustomer = await getBillingCustomerContext();
  const result = await fetchLighthouseResult(
    "https://example.com",
    "page-123",
    "mobile",
    billingCustomer,
  );

  console.log("Performance Score:", result.result.performanceScore);
  console.log("LCP (ms):", result.result.lcpMs);
}

// Selecting a representative URL sample
import { selectLighthouseSample } from "@/server/lib/audit/lighthouse";

const pages = [
  { url: "https://example.com", statusCode: 200 },
  { url: "https://example.com/about", statusCode: 200 },
  { url: "https://example.com/products/item", statusCode: 200 },
];

const sample = selectLighthouseSample(pages, "https://example.com", "auto");
console.log(sample); // Array of up to 10 URLs covering distinct patterns

Summary

• Intelligent sampling – selectLighthouseSample() limits API costs by auditing up to 10 strategic URLs including the homepage and unique page templates.
• Type-safe API layer – The DataForSEO integration uses strongly-typed request builders and Zod schemas to prevent runtime errors.
• Comprehensive metrics – The parser extracts four category scores and four Core Web Vitals metrics, converting all values to consistent percentage formats.
• Cloud-native storage – Results optionally persist to Cloudflare R2 with associated metadata keys for downstream analysis.

Frequently Asked Questions

Which DataForSEO endpoint does OpenSEO use for Lighthouse data?

OpenSEO submits POST requests to DataForSEO's /lighthouse/live/json endpoint via the low-level client in src/server/lib/dataforseo/lighthouse.ts. This endpoint accepts OnPageLighthouseLiveJsonRequestInfo payloads containing the target URL and device strategy (mobile, desktop, or auto).

How does OpenSEO validate Lighthouse API responses?

The system uses Zod schemas defined in src/server/lib/dataforseoLighthousePayload.ts to validate the raw JSON structure before processing. The dataforseoLighthouseResponseSchema enforces type safety on nested audit objects, ensuring that malformed responses fail fast before entering the storage pipeline.

What specific metrics does OpenSEO extract from Lighthouse reports?

According to the source code in src/server/lib/dataforseoLighthousePayload.ts, the parser extracts category scores for performance, accessibility, best-practices, and SEO. It also captures Core Web Vitals including Largest Contentful Paint (LCP), Cumulative Layout Shift (CLS), Interaction to Next Paint (INP), and Server Response Time.

Where are Lighthouse results stored after fetching?

The storeLighthouseResult() function in src/server/lib/audit/lighthouse.ts persists normalized data to Cloudflare R2 object storage when configured. The function returns a LighthouseResult containing storage keys and payload size information, enabling retrieval of full audit details without repeatedly hitting the DataForSEO API.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}