What Kind of Data Does Open‑SEO Process? A Complete Technical Breakdown

Open‑SEO processes three distinct layers of SEO data—user inputs (seed keywords and project settings), external API responses (DataForSEO, Google Search Console, and Ahrefs), and derived analytics (crawl results, Lighthouse scores, and rank‑tracking snapshots)—validating all streams through Zod schemas before persistence in a PostgreSQL database.

The every-app/open-seo repository is a full‑stack SEO platform that automates the ingestion, enrichment, and analysis of search engine data. Understanding exactly what kind of data Open‑SEO handles is critical for developers extending its Model Context Protocol (MCP) tools, troubleshooting pipeline failures, or auditing data privacy compliance.

External API Data: Third‑Party SEO Intelligence

Open‑SEO acts as an aggregator for multiple third‑party SEO data providers, normalizing disparate API responses into a unified internal schema.

Keyword Research Metrics

The platform pulls keyword volume, CPC, difficulty scores, and competition metrics from the DataForSEO API. In src/server/mcp/tools/research-keywords.ts, the research-keywords tool wraps the KeywordResearchService, calling DataForSEO’s keyword‑ideas and keyword‑metrics endpoints. These calls return seed keywords and their associated metrics, which are then validated against Zod schemas defined in src/types/schemas/keywords.ts before storage.

Live SERP Results

For competitive analysis, Open‑SEO fetches live Google Search Engine Results Page (SERP) data—including organic listings, featured snippets, ads, and local pack entries. The get-serp-results.ts MCP tool in src/server/mcp/tools/ utilizes the fetchSerpResults helper to query DataForSEO’s serp‑live API (with Google Ads API fallback), storing the raw JSON for downstream analysis.

Backlink and Authority Data

Backlink profiles are retrieved through DataForSEO’s backlinks endpoint via src/serverFunctions/backlinks.ts, capturing referring domains, anchor text, and link types. Domain authority metrics (Domain Rating and URL Rating from Ahrefs) are processed through src/serverFunctions/ahrefs.ts, where the fetchDomainRating function normalizes Ahrefs KPIs for UI display.

Google Search Console (GSC) Integration

Organic performance data—impressions, clicks, CTR, and average position per query and page—flows from the official Google Search Console API. The GscService class in src/serverFunctions/gsc.ts orchestrates synchronization into the gsc schema, enabling historical trend analysis.

Derived Internal Data: Platform‑Generated Analytics

Beyond external ingestion, Open‑SEO generates its own datasets through automated crawling and scheduled checks.

Rank Tracking Snapshots

The RankCheckWorkflow in src/server/workflows/RankCheckWorkflow.ts executes batch jobs that fetch SERP snapshots for every tracked keyword/device pair. Using DataForSEO’s rank‑check endpoint, it records the periodic position of each keyword on Google (desktop and mobile), persisting results to the rank_tracking table via logic in src/serverFunctions/rank-tracking.ts.

Site Audit Crawl Data

The crawling engine in src/server/workflows/siteAuditWorkflowCrawl.ts performs HTTP fetches against target URLs, classifying responses into fetch classes (ok, blocked, error) and capturing HTTP status codes and page titles. Optionally, it triggers Lighthouse audits to record performance scores, storing all audit artifacts in the audit schema managed by src/serverFunctions/audit.ts.

User‑Supplied Input Data

Saved Keywords and Metadata

Users can manually curate keyword lists with custom tags and notes. The MCP tools save_keywords and list_saved_keywords—implemented in src/server/mcp/tools/save-keywords.ts and src/server/mcp/tools/list-saved-keywords.ts—manage this cached metric metadata without consuming external API credits, reading and writing directly to the saved_keywords table.

Project Configuration and Billing

Project settings, user permissions, usage limits, and credit balances are handled by BillingService and ProjectService. These services, exposed through src/serverFunctions/billing.ts and src/serverFunctions/projects.ts, enforce audit‑limit caps and track internal telemetry data for quota management.

Data Validation and Storage Architecture

All data streams pass through strict Zod schema validation before persistence. Keyword payloads are validated against definitions in src/types/schemas/keywords.ts, while AI‑search related data uses src/types/schemas/ai-search.ts. The central database schema defined in src/db/schema.ts (using Drizzle ORM) structures tables for keywords, audits, rank tracking, and GSC data, ensuring type safety across the full stack.

Practical Code Examples

// Fetching keyword metrics from DataForSEO
import { KeywordResearchService } from "@/server/features/keywords/services/KeywordResearchService";

const seeds = ["seo tools", "open source seo"];
const result = await KeywordResearchService.researchKeywords(seeds, {
  maxKeywordsPerSeed: 150,
});
console.log(result); // Array of objects with volume, difficulty, CPC

// Executing a site audit crawl
import { crawlSite } from "@/server/workflows/siteAuditWorkflowCrawl";

await crawlSite({
  projectId: "proj_123",
  url: "https://example.com",
  userId: "user_456",
});

// Retrieving user-saved keywords with filters
import { listSavedKeywordsTool } from "@/server/mcp/tools/list-saved-keywords";

const { rows } = await listSavedKeywordsTool({ 
  projectId: "proj_123", 
  search: "seo", 
  tags: ["blog"] 
});
console.log(rows); // Cached keyword metadata

Summary

• Open‑SEO ingests three data layers: user inputs (seeds and configurations), external API data (DataForSEO, GSC, Ahrefs), and derived internal analytics (crawl results, rank snapshots).
• Key processing modules include KeywordResearchService for keyword intelligence, RankCheckWorkflow for position tracking, and siteAuditWorkflowCrawl for technical SEO audits.
• Validation occurs at the boundary using Zod schemas (src/types/schemas/keywords.ts) before data reaches the Drizzle ORM layer (src/db/schema.ts).
• No‑credit operations like save_keywords allow users to store metadata without triggering external API billing.

Frequently Asked Questions

What external APIs does Open‑SEO integrate with?

Open‑SEO primarily integrates with the DataForSEO API for keyword research, SERP results, and backlink data. It also connects to the Google Search Console API for organic performance metrics, the Ahrefs API for domain authority scores, and optionally the Google Ads API as a fallback for SERP data.

How does Open‑SEO validate incoming data before storage?

All incoming data is validated using Zod schemas located in src/types/schemas/. For example, keyword payloads are checked against keywords.ts schemas, while audit data schemas ensure HTTP status codes and Lighthouse scores conform to expected types before being written to the PostgreSQL database via Drizzle ORM.

Can Open‑SEO process data without consuming external API credits?

Yes. Operations involving saved keywords—such as tagging, note-taking, or listing previously researched keywords—are handled entirely within the internal database. The save_keywords and list_saved_keywords MCP tools in src/server/mcp/tools/ interact only with the saved_keywords table, consuming no DataForSEO or Ahrefs credits.

Where is crawl and audit data stored after processing?

Technical SEO audit data—including HTTP response classes, page titles, and Lighthouse performance scores—is stored in the audit schema. The crawling workflow in src/server/workflows/siteAuditWorkflowCrawl.ts persists this data through src/serverFunctions/audit.ts, making it available for historical analysis and regression detection.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}