# How to Use OpenSEO for SEO Analysis: A Complete Technical Guide

> Learn how to use OpenSEO for powerful SEO analysis. This technical guide covers Cloudflare Workers crawling, D1 database, and AI-compatible workflows for scalable insights.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: how-to-guide
- Published: 2026-08-06

---

**OpenSEO is a modern, open-source SEO platform that combines Cloudflare Workers-based crawling, D1 database storage, and MCP (Model-Context-Protocol) endpoints to deliver scalable, AI-compatible SEO analysis workflows.**

OpenSEO, available in the `every-app/open-seo` repository, provides a stateless, self-hostable alternative to traditional SEO tools. This guide covers how to use open-seo for SEO analysis by leveraging its serverless architecture, incremental storage system, and programmatic API layer.

## Architecture Overview

OpenSEO operates through three integrated layers that handle everything from page fetching to AI-ready data consumption.

### Data Collection Layer

The **crawler** runs as a Cloudflare Worker configured in [`vite-plugin-lean-worker-bundle.ts`](https://github.com/every-app/open-seo/blob/main/vite-plugin-lean-worker-bundle.ts). It fetches pages, robots.txt, and sitemaps while extracting raw signals including status codes, titles, meta tags, and link graphs. The worker architecture ensures the service remains fast, stateless, and horizontally scalable without long-running servers.

### Processing and Storage Layer

Each crawl batch writes normalized data to **Cloudflare D1** (SQLite-compatible) via the `audit_pages` and `audit_links` tables. The `SiteAuditWorkflow` class in [`src/server/workflows/SiteAuditWorkflow.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/SiteAuditWorkflow.ts) manages this process, scoping a per-request Postgres/D1 client and enforcing step budgets to respect Cloudflare Workers limits. This incremental approach uses one `db.batch` per crawl batch, keeping the link graph persisted without hitting the 1 MiB step-state cap.

### MCP API Layer

The server exposes a **Model-Context-Protocol** transport layer in [`src/server/mcp/transport.ts`](https://github.com/every-app/open-seo/blob/main/src/server/mcp/transport.ts) built on the `@modelcontextprotocol/sdk`. This lightweight RPC layer operates over HTTP-SSE or JSON-RPC, allowing AI agents like Claude Code, OpenClaw, or Hermes to retrieve SEO data, run issue detectors, and receive actionable remediation instructions. The MCP tools in `src/server/mcp/tools/*` package each issue with human-readable fix instructions, making them directly actionable for automated agents.

## Self-Hosting OpenSEO

Running open-seo locally requires Docker and a DataForSEO API key for enhanced crawling capabilities.

### Docker Setup

Clone the repository and launch the containerized environment:

```bash
git clone https://github.com/every-app/open-seo.git
cd open-seo
cp .env.example .env
docker compose up -d

```

The `Dockerfile.selfhost` and [`compose.yaml`](https://github.com/every-app/open-seo/blob/main/compose.yaml) configure the Workers runtime, D1 database, and Vite dev server. See [`docs/SELF_HOSTING_DOCKER.md`](https://github.com/every-app/open-seo/blob/main/docs/SELF_HOSTING_DOCKER.md) for environment-specific configurations and telemetry controls.

### Configuring DataForSEO

Provide your **DataForSEO API key** (base64-encoded `email:password`) in the `.env` file:

```bash
sed -i 's|DATAFORSEO_API_KEY=.*|DATAFORSEO_API_KEY=YOUR_BASE64_KEY|' .env

```

The server-side client in [`src/server/lib/dataforseo/shared.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/shared.ts) handles authentication for both the crawler and SERP/rank-tracking functions. Refer to [`docs/DATAFORSEO_API_KEY.md`](https://github.com/every-app/open-seo/blob/main/docs/DATAFORSEO_API_KEY.md) for key generation instructions.

## Running SEO Audits

Initiate crawls via HTTP API or the React-based UI located in the `web/` directory.

### HTTP API Method

Trigger a site audit by calling the `/api/crawl` endpoint:

```bash
curl -X POST "http://localhost:3001/api/crawl" \
     -H "Content-Type: application/json" \
     -d '{"url":"https://example.com"}'

```

This instantiates a `SiteAuditWorkflow` that drives the Workers crawler, stores pages and links in D1, and executes the **issue engine** to detect duplicate titles, broken links, and thin content.

### Understanding the Workflow

The `SiteAuditWorkflow` orchestrates the entire process:
- Spawns Cloudflare Worker instances for parallel fetching
- Batches writes to the `audit_pages` and `audit_links` tables
- Runs issue detection algorithms against the normalized dataset
- Maintains state within Cloudflare's step limits

## Consuming Analysis Results

OpenSEO exposes results through both a web interface and the MCP API.

### UI-Based Access

The React client provides visual inspection of audit results, including sortable issue lists and link graph visualizations. Export functionality generates CSV files for offline analysis.

### Programmatic MCP Access

For automation and AI integration, use the MCP client:

```typescript
import { Client } from "@modelcontextprotocol/sdk/client";

const client = new Client({ name: "my-app", version: "1.0.0" });
await client.connect({ url: "http://localhost:3001/mcp" });

// Start an audit
const { auditId } = await client.call("run_site_audit", {
  url: "https://example.com",
});

// Poll for completion
let status = "running";
while (status === "running") {
  const resp = await client.call("get_audit_status", { auditId });
  status = resp.status;
  await new Promise(r => setTimeout(r, 1000));
}

// Retrieve high-severity issues
const issues = await client.call("get_audit_issues", {
  auditId,
  severity: "high",
});
console.log(issues);

```

Available MCP methods include `run_site_audit`, `get_audit_status`, and `get_audit_issues`, each returning structured data with remediation guidance.

### Exporting Data

Download audit results as CSV via command line:

```bash
curl "http://localhost:3001/api/audit/export?auditId=$AUDIT_ID" -o audit.csv

```

## Scheduled Audits and Rank Tracking

Enable continuous monitoring by extending the `RankCheckWorkflow` in [`src/server/workflows/RankCheckWorkflow.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/RankCheckWorkflow.ts). This scheduler stores deltas between runs, enabling trend analysis and regression detection over time.

## Summary

- **OpenSEO** uses Cloudflare Workers for stateless, scalable crawling that avoids long-running server costs
- **D1 storage** in `audit_pages` and `audit_links` tables provides cost-effective persistence at approximately $0.30 per 1,000 links
- **SiteAuditWorkflow** manages the entire pipeline from crawling to issue detection while respecting Cloudflare step limits
- **MCP transport** in [`src/server/mcp/transport.ts`](https://github.com/every-app/open-seo/blob/main/src/server/mcp/transport.ts) enables AI agents to consume SEO data programmatically
- **Self-hosting** via Docker gives you complete control over crawler user-agents, politeness settings, and block-handling logic

## Frequently Asked Questions

### What makes OpenSEO different from traditional SEO tools?

Unlike SaaS SEO platforms, OpenSEO is fully self-hostable and stateless. By running on Cloudflare Workers and D1, you control the infrastructure while paying only for storage and compute used. The MCP API layer also makes it uniquely suited for AI-driven workflows, allowing agents to directly query audit data and receive fix instructions.

### Do I need a DataForSEO API key to use OpenSEO?

No, the open-source crawler works independently. However, configuring a **DataForSEO** API key (base64 `email:password` in `.env`) enhances capabilities with SERP data and rank tracking via [`src/server/lib/dataforseo/shared.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/dataforseo/shared.ts). The core crawling and analysis functions operate without third-party APIs.

### How does OpenSEO handle large sites without hitting Cloudflare limits?

The `SiteAuditWorkflow` implements **incremental D1 writes** using `db.batch` operations per crawl batch. This design persists the link graph incrementally rather than storing it in step state, avoiding the 1 MiB step-state cap while maintaining full crawl continuity.

### Can I integrate OpenSEO with my existing AI agents or automation tools?

Yes. The **Model-Context-Protocol** implementation in [`src/server/mcp/transport.ts`](https://github.com/every-app/open-seo/blob/main/src/server/mcp/transport.ts) exposes standardized endpoints that any MCP-compatible client can consume. This includes methods to trigger audits, check status, and retrieve issues with built-in remediation guidance, making it ideal for CI/CD pipelines or AI agent workflows.