How to Use OpenSEO for SEO Analysis: A Complete Technical Guide

OpenSEO is a modern, open-source SEO platform that combines Cloudflare Workers-based crawling, D1 database storage, and MCP (Model-Context-Protocol) endpoints to deliver scalable, AI-compatible SEO analysis workflows.

OpenSEO, available in the every-app/open-seo repository, provides a stateless, self-hostable alternative to traditional SEO tools. This guide covers how to use open-seo for SEO analysis by leveraging its serverless architecture, incremental storage system, and programmatic API layer.

Architecture Overview

OpenSEO operates through three integrated layers that handle everything from page fetching to AI-ready data consumption.

Data Collection Layer

The crawler runs as a Cloudflare Worker configured in vite-plugin-lean-worker-bundle.ts. It fetches pages, robots.txt, and sitemaps while extracting raw signals including status codes, titles, meta tags, and link graphs. The worker architecture ensures the service remains fast, stateless, and horizontally scalable without long-running servers.

Processing and Storage Layer

Each crawl batch writes normalized data to Cloudflare D1 (SQLite-compatible) via the audit_pages and audit_links tables. The SiteAuditWorkflow class in src/server/workflows/SiteAuditWorkflow.ts manages this process, scoping a per-request Postgres/D1 client and enforcing step budgets to respect Cloudflare Workers limits. This incremental approach uses one db.batch per crawl batch, keeping the link graph persisted without hitting the 1 MiB step-state cap.

MCP API Layer

The server exposes a Model-Context-Protocol transport layer in src/server/mcp/transport.ts built on the @modelcontextprotocol/sdk. This lightweight RPC layer operates over HTTP-SSE or JSON-RPC, allowing AI agents like Claude Code, OpenClaw, or Hermes to retrieve SEO data, run issue detectors, and receive actionable remediation instructions. The MCP tools in src/server/mcp/tools/* package each issue with human-readable fix instructions, making them directly actionable for automated agents.

Self-Hosting OpenSEO

Running open-seo locally requires Docker and a DataForSEO API key for enhanced crawling capabilities.

Docker Setup

Clone the repository and launch the containerized environment:

git clone https://github.com/every-app/open-seo.git
cd open-seo
cp .env.example .env
docker compose up -d

The Dockerfile.selfhost and compose.yaml configure the Workers runtime, D1 database, and Vite dev server. See docs/SELF_HOSTING_DOCKER.md for environment-specific configurations and telemetry controls.

Configuring DataForSEO

Provide your DataForSEO API key (base64-encoded email:password) in the .env file:

sed -i 's|DATAFORSEO_API_KEY=.*|DATAFORSEO_API_KEY=YOUR_BASE64_KEY|' .env

The server-side client in src/server/lib/dataforseo/shared.ts handles authentication for both the crawler and SERP/rank-tracking functions. Refer to docs/DATAFORSEO_API_KEY.md for key generation instructions.

Running SEO Audits

Initiate crawls via HTTP API or the React-based UI located in the web/ directory.

HTTP API Method

Trigger a site audit by calling the /api/crawl endpoint:

curl -X POST "http://localhost:3001/api/crawl" \
     -H "Content-Type: application/json" \
     -d '{"url":"https://example.com"}'

This instantiates a SiteAuditWorkflow that drives the Workers crawler, stores pages and links in D1, and executes the issue engine to detect duplicate titles, broken links, and thin content.

Understanding the Workflow

The SiteAuditWorkflow orchestrates the entire process:

• Spawns Cloudflare Worker instances for parallel fetching
• Batches writes to the audit_pages and audit_links tables
• Runs issue detection algorithms against the normalized dataset
• Maintains state within Cloudflare's step limits

Consuming Analysis Results

OpenSEO exposes results through both a web interface and the MCP API.

UI-Based Access

The React client provides visual inspection of audit results, including sortable issue lists and link graph visualizations. Export functionality generates CSV files for offline analysis.

Programmatic MCP Access

For automation and AI integration, use the MCP client:

import { Client } from "@modelcontextprotocol/sdk/client";

const client = new Client({ name: "my-app", version: "1.0.0" });
await client.connect({ url: "http://localhost:3001/mcp" });

// Start an audit
const { auditId } = await client.call("run_site_audit", {
  url: "https://example.com",
});

// Poll for completion
let status = "running";
while (status === "running") {
  const resp = await client.call("get_audit_status", { auditId });
  status = resp.status;
  await new Promise(r => setTimeout(r, 1000));
}

// Retrieve high-severity issues
const issues = await client.call("get_audit_issues", {
  auditId,
  severity: "high",
});
console.log(issues);

Available MCP methods include run_site_audit, get_audit_status, and get_audit_issues, each returning structured data with remediation guidance.

Exporting Data

Download audit results as CSV via command line:

curl "http://localhost:3001/api/audit/export?auditId=$AUDIT_ID" -o audit.csv

Scheduled Audits and Rank Tracking

Enable continuous monitoring by extending the RankCheckWorkflow in src/server/workflows/RankCheckWorkflow.ts. This scheduler stores deltas between runs, enabling trend analysis and regression detection over time.

Summary

• OpenSEO uses Cloudflare Workers for stateless, scalable crawling that avoids long-running server costs
• D1 storage in audit_pages and audit_links tables provides cost-effective persistence at approximately $0.30 per 1,000 links
• SiteAuditWorkflow manages the entire pipeline from crawling to issue detection while respecting Cloudflare step limits
• MCP transport in src/server/mcp/transport.ts enables AI agents to consume SEO data programmatically
• Self-hosting via Docker gives you complete control over crawler user-agents, politeness settings, and block-handling logic

Frequently Asked Questions

What makes OpenSEO different from traditional SEO tools?

Unlike SaaS SEO platforms, OpenSEO is fully self-hostable and stateless. By running on Cloudflare Workers and D1, you control the infrastructure while paying only for storage and compute used. The MCP API layer also makes it uniquely suited for AI-driven workflows, allowing agents to directly query audit data and receive fix instructions.

Do I need a DataForSEO API key to use OpenSEO?

No, the open-source crawler works independently. However, configuring a DataForSEO API key (base64 email:password in .env) enhances capabilities with SERP data and rank tracking via src/server/lib/dataforseo/shared.ts. The core crawling and analysis functions operate without third-party APIs.

How does OpenSEO handle large sites without hitting Cloudflare limits?

The SiteAuditWorkflow implements incremental D1 writes using db.batch operations per crawl batch. This design persists the link graph incrementally rather than storing it in step state, avoiding the 1 MiB step-state cap while maintaining full crawl continuity.

Can I integrate OpenSEO with my existing AI agents or automation tools?

Yes. The Model-Context-Protocol implementation in src/server/mcp/transport.ts exposes standardized endpoints that any MCP-compatible client can consume. This includes methods to trigger audits, check status, and retrieve issues with built-in remediation guidance, making it ideal for CI/CD pipelines or AI agent workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}