How OpenSEO Handles On-Page SEO Analysis: Architecture and Implementation

OpenSEO performs on-page SEO analysis through a multi-stage pipeline that crawls URLs, executes Lighthouse audits, validates HTML against a declarative issue catalog, and aggregates site-wide patterns via multipage checks.

The open-source OpenSEO project (every-app/open-seo) provides a comprehensive engine for diagnosing page-level search optimization issues. Its on-page SEO analysis system combines automated crawling with a declarative issue detection framework to identify problems ranging from missing title tags to duplicate content signatures. The architecture separates discovery, validation, and persistence into distinct phases, enabling scalable audits across thousands of pages.

The Site Audit Workflow Architecture

The SiteAuditWorkflow class orchestrates the entire on-page SEO analysis process. According to the source code, this workflow coordinates four primary stages: URL discovery, page fetching and processing, Lighthouse execution, and multipage validation. Each stage feeds data into a centralized repository that tracks progress and stores granular issue details.

The workflow begins by respecting robots.txt policies and URL crawlability rules before fetching page content. After HTML extraction, the system branches into two parallel tracks: Lighthouse performance analysis and declarative issue scanning. Finally, runMultipageChecks aggregates individual page data to detect site-wide patterns such as orphan pages or redirect chains.

Stage 1: Crawl Discovery and URL Policy

Before any on-page analysis occurs, the workflow must discover and validate target URLs. The system uses discoverUrls to extract links from the start page, then filters them through parseRobotsTxt and isCrawlableUrl to ensure compliance with crawling policies. This prevents the audit from wasting resources on disallowed paths while ensuring comprehensive coverage of accessible content.

The discovery phase writes discovered URLs to a queue that worker processes consume. Each URL carries metadata about its discovery depth and parent page, enabling the reconstruction of site architecture during the multipage analysis phase.

Stage 2: Lighthouse Execution for Core Web Vitals

For every successfully fetched page, OpenSEO executes runLighthouse to capture performance, accessibility, and core web vitals metrics. This integration provides objective measurements for LCP (Largest Contentful Paint), CLS (Cumulative Layout Shift), and other technical SEO factors that influence search rankings.

The Lighthouse results are persisted immediately via AuditRepository.insertLighthouseResults, which stores the JSON output alongside page metadata. This separation of performance data from content issues allows developers to query specific metric thresholds independently of HTML validation results.

Stage 3: Declarative On-Page Issue Detection

The core of OpenSEO's on-page analysis lies in its declarative issue catalog. Rather than hardcoding validation logic, the system maps HTML content against a structured registry of SEO issues defined in src/shared/audit-issues.ts. This approach makes the audit engine extensible without modifying core workflow code.

The Central Issue Registry

The audit-issues.ts file serves as the single source of truth for on-page SEO problems. Each entry contains a unique identifier, human-readable title, detailed description, and remediation guidance. The catalog covers critical on-page elements including:

• Missing title tag: Detects pages lacking a <title> element in the document head
• Duplicate title: Identifies when multiple URLs share identical title tag content
• Missing H1 heading: Flags pages without a primary H1 element or with multiple H1 tags
• Images missing alt text: Reports <img> elements lacking descriptive alt attributes
• Thin content: Identifies pages with insufficient text content relative to template markup

Issues are categorized by severity and SEO impact, allowing the workflow to prioritize critical problems over informational suggestions.

HTML Parsing and Issue Mapping

The page-reporters.ts module implements the logic that transforms raw HTML into issue reports. This parser extracts semantic elements—title tags, meta descriptions, heading hierarchies, image attributes, and canonical links—then validates them against the audit-issues.ts registry.

For example, when analyzing heading structure, the reporter checks for the presence of exactly one H1 tag per page. If the parser finds zero or multiple H1 elements, it generates an issue row referencing the "missing-h1" or "multiple-h1" definition from the shared catalog. This mapping ensures consistent messaging across all audit results.

// Example: Triggering a site audit workflow
import { SiteAuditWorkflow } from '@/server/workflows/SiteAuditWorkflow';

await SiteAuditWorkflow.run({
  projectId: 'proj_123',
  startUrl: 'https://example.com',
  maxPages: 500,
  crawlDepth: 3
});

Stage 4: Site-Wide Multipage Validation

After individual pages are processed, OpenSEO executes runMultipageChecks via src/server/lib/audit/issues/multipage.ts. This stage analyzes aggregate data to detect patterns invisible at the single-page level, including:

• Orphan pages: URLs with no internal incoming links
• Redirect chains: Sequences of multiple redirects between discovery and final content
• Duplicate content clusters: Pages with substantially similar body content but different URLs
• Broken internal links: References to 404 or 5xx pages found during crawling

These checks require comparing the entire crawled dataset, making them computationally distinct from the streaming page-level analysis performed earlier in the pipeline.

Data Persistence and Audit State Management

Throughout the audit, progress tracking occurs via AuditRepository.updateAuditProgress, which writes to a KV store and relational database. The repository maintains three distinct data structures:

1. Audit metadata: Current phase, pages processed, and completion percentage
2. Issue rows: Individual instances of problems detected (e.g., "Page X is missing meta description")
3. Lighthouse results: Full JSON reports for each analyzed URL

This architecture supports resumable audits—if a workflow interrupts, the system can restart from the last persisted state rather than recrawling completed pages.

// Example: Extending the issue catalog with custom rules
import { auditIssues } from '@/shared/audit-issues';

auditIssues['schema-missing'] = {
  title: 'Missing Schema.org Structured Data',
  description: 
    'The page does not contain JSON-LD or Microdata markup for primary entities.',
  remediation: 
    'Add appropriate Schema.org markup in the <head> or body to enhance SERP appearance.',
  severity: 'warning'
};

Summary

• OpenSEO's on-page SEO analysis runs through a four-stage pipeline: URL discovery, Lighthouse execution, declarative issue detection, and multipage validation.
• Issue detection relies on a centralized registry in src/shared/audit-issues.ts that defines titles, descriptions, and fixes for common SEO problems.
• Page-level reporters extract HTML elements and validate them against the issue catalog, while multipage checks identify site-wide patterns like orphan pages and duplicate content.
• All results persist through AuditRepository, which stores Lighthouse metrics, individual issue instances, and audit progress state.

Frequently Asked Questions

How does OpenSEO detect specific on-page issues like missing title tags?

OpenSEO detects missing title tags through the HTML parser in page-reporters.ts, which scans the <head> section of each crawled page for <title> elements. When the parser finds no title tag or an empty one, it references the audit-issues.ts catalog to generate a standardized report containing the issue title "Missing title tag," a description of the SEO impact, and specific remediation steps. The system uses the same pattern for meta descriptions and canonical tags.

What distinguishes page-level checks from multipage checks in OpenSEO?

Page-level checks examine individual HTML documents for semantic structure and content quality issues—such as missing alt text or improper heading hierarchies—during the initial crawl phase. Multipage checks, implemented in multipage.ts, run after all individual pages are processed and analyze relationships between URLs, detecting site-wide problems like orphan pages, redirect chains, and duplicate content across multiple URLs that would be invisible when viewing pages in isolation.

Can developers add custom on-page SEO rules to OpenSEO?

Yes, developers can extend the audit engine by adding entries to the auditIssues object exported from src/shared/audit-issues.ts. Each new issue requires a unique key, title, description, and remediation guidance. After defining the issue metadata, implement the detection logic in page-reporters.ts to scan for the specific HTML pattern. The declarative architecture ensures new rules automatically integrate with the reporting UI and persistence layer without modifying the core workflow.

Where does OpenSEO store audit results during the analysis process?

OpenSEO stores audit results in two locations: a KV store for transient progress tracking and a relational database for permanent records. The AuditRepository.updateAuditProgress method writes phase completion percentages and metadata to the KV store for real-time monitoring, while AuditRepository.insertLighthouseResults and issue-specific insert methods write granular findings to tables including audit_issues and lighthouse_results. This dual-storage approach supports both resumable workflows and long-term historical analysis.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}