# What Kind of Data Does Open‑SEO Process? A Complete Technical Breakdown

> Explore the SEO data processed by Open-SEO. Learn about user inputs, API responses from DataForSEO and Ahrefs, and derived analytics like crawl results and Lighthouse scores for comprehensive SEO insights.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: deep-dive
- Published: 2026-07-26

---

**Open‑SEO processes three distinct layers of SEO data—user inputs (seed keywords and project settings), external API responses (DataForSEO, Google Search Console, and Ahrefs), and derived analytics (crawl results, Lighthouse scores, and rank‑tracking snapshots)—validating all streams through Zod schemas before persistence in a PostgreSQL database.**

The `every-app/open-seo` repository is a full‑stack SEO platform that automates the ingestion, enrichment, and analysis of search engine data. Understanding exactly what kind of data Open‑SEO handles is critical for developers extending its Model Context Protocol (MCP) tools, troubleshooting pipeline failures, or auditing data privacy compliance.

## External API Data: Third‑Party SEO Intelligence

Open‑SEO acts as an aggregator for multiple third‑party SEO data providers, normalizing disparate API responses into a unified internal schema.

### Keyword Research Metrics

The platform pulls **keyword volume**, **CPC**, **difficulty scores**, and **competition metrics** from the DataForSEO API. In [`src/server/mcp/tools/research-keywords.ts`](https://github.com/every-app/open-seo/blob/main/src/server/mcp/tools/research-keywords.ts), the `research-keywords` tool wraps the `KeywordResearchService`, calling DataForSEO’s *keyword‑ideas* and *keyword‑metrics* endpoints. These calls return seed keywords and their associated metrics, which are then validated against Zod schemas defined in [`src/types/schemas/keywords.ts`](https://github.com/every-app/open-seo/blob/main/src/types/schemas/keywords.ts) before storage.

### Live SERP Results

For competitive analysis, Open‑SEO fetches live Google Search Engine Results Page (SERP) data—including organic listings, featured snippets, ads, and local pack entries. The [`get-serp-results.ts`](https://github.com/every-app/open-seo/blob/main/get-serp-results.ts) MCP tool in `src/server/mcp/tools/` utilizes the `fetchSerpResults` helper to query DataForSEO’s *serp‑live* API (with Google Ads API fallback), storing the raw JSON for downstream analysis.

### Backlink and Authority Data

Backlink profiles are retrieved through DataForSEO’s *backlinks* endpoint via [`src/serverFunctions/backlinks.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/backlinks.ts), capturing **referring domains**, **anchor text**, and **link types**. Domain authority metrics (Domain Rating and URL Rating from Ahrefs) are processed through [`src/serverFunctions/ahrefs.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/ahrefs.ts), where the `fetchDomainRating` function normalizes Ahrefs KPIs for UI display.

### Google Search Console (GSC) Integration

Organic performance data—**impressions**, **clicks**, **CTR**, and **average position** per query and page—flows from the official Google Search Console API. The `GscService` class in [`src/serverFunctions/gsc.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/gsc.ts) orchestrates synchronization into the `gsc` schema, enabling historical trend analysis.

## Derived Internal Data: Platform‑Generated Analytics

Beyond external ingestion, Open‑SEO generates its own datasets through automated crawling and scheduled checks.

### Rank Tracking Snapshots

The `RankCheckWorkflow` in [`src/server/workflows/RankCheckWorkflow.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/RankCheckWorkflow.ts) executes batch jobs that fetch SERP snapshots for every tracked keyword/device pair. Using DataForSEO’s rank‑check endpoint, it records the **periodic position** of each keyword on Google (desktop and mobile), persisting results to the `rank_tracking` table via logic in [`src/serverFunctions/rank-tracking.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/rank-tracking.ts).

### Site Audit Crawl Data

The crawling engine in [`src/server/workflows/siteAuditWorkflowCrawl.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/siteAuditWorkflowCrawl.ts) performs HTTP fetches against target URLs, classifying responses into **fetch classes** (ok, blocked, error) and capturing **HTTP status codes** and **page titles**. Optionally, it triggers Lighthouse audits to record **performance scores**, storing all audit artifacts in the `audit` schema managed by [`src/serverFunctions/audit.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/audit.ts).

## User‑Supplied Input Data

### Saved Keywords and Metadata

Users can manually curate keyword lists with custom tags and notes. The MCP tools `save_keywords` and `list_saved_keywords`—implemented in [`src/server/mcp/tools/save-keywords.ts`](https://github.com/every-app/open-seo/blob/main/src/server/mcp/tools/save-keywords.ts) and [`src/server/mcp/tools/list-saved-keywords.ts`](https://github.com/every-app/open-seo/blob/main/src/server/mcp/tools/list-saved-keywords.ts)—manage this **cached metric metadata** without consuming external API credits, reading and writing directly to the `saved_keywords` table.

### Project Configuration and Billing

Project settings, user permissions, usage limits, and credit balances are handled by `BillingService` and `ProjectService`. These services, exposed through [`src/serverFunctions/billing.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/billing.ts) and [`src/serverFunctions/projects.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/projects.ts), enforce audit‑limit caps and track internal telemetry data for quota management.

## Data Validation and Storage Architecture

All data streams pass through strict **Zod schema validation** before persistence. Keyword payloads are validated against definitions in [`src/types/schemas/keywords.ts`](https://github.com/every-app/open-seo/blob/main/src/types/schemas/keywords.ts), while AI‑search related data uses [`src/types/schemas/ai-search.ts`](https://github.com/every-app/open-seo/blob/main/src/types/schemas/ai-search.ts). The central database schema defined in [`src/db/schema.ts`](https://github.com/every-app/open-seo/blob/main/src/db/schema.ts) (using Drizzle ORM) structures tables for keywords, audits, rank tracking, and GSC data, ensuring type safety across the full stack.

### Practical Code Examples

```typescript
// Fetching keyword metrics from DataForSEO
import { KeywordResearchService } from "@/server/features/keywords/services/KeywordResearchService";

const seeds = ["seo tools", "open source seo"];
const result = await KeywordResearchService.researchKeywords(seeds, {
  maxKeywordsPerSeed: 150,
});
console.log(result); // Array of objects with volume, difficulty, CPC

```

```typescript
// Executing a site audit crawl
import { crawlSite } from "@/server/workflows/siteAuditWorkflowCrawl";

await crawlSite({
  projectId: "proj_123",
  url: "https://example.com",
  userId: "user_456",
});

```

```typescript
// Retrieving user-saved keywords with filters
import { listSavedKeywordsTool } from "@/server/mcp/tools/list-saved-keywords";

const { rows } = await listSavedKeywordsTool({ 
  projectId: "proj_123", 
  search: "seo", 
  tags: ["blog"] 
});
console.log(rows); // Cached keyword metadata

```

## Summary

- **Open‑SEO ingests three data layers**: user inputs (seeds and configurations), external API data (DataForSEO, GSC, Ahrefs), and derived internal analytics (crawl results, rank snapshots).
- **Key processing modules** include `KeywordResearchService` for keyword intelligence, `RankCheckWorkflow` for position tracking, and `siteAuditWorkflowCrawl` for technical SEO audits.
- **Validation occurs at the boundary** using Zod schemas ([`src/types/schemas/keywords.ts`](https://github.com/every-app/open-seo/blob/main/src/types/schemas/keywords.ts)) before data reaches the Drizzle ORM layer ([`src/db/schema.ts`](https://github.com/every-app/open-seo/blob/main/src/db/schema.ts)).
- **No‑credit operations** like `save_keywords` allow users to store metadata without triggering external API billing.

## Frequently Asked Questions

### What external APIs does Open‑SEO integrate with?

Open‑SEO primarily integrates with the **DataForSEO API** for keyword research, SERP results, and backlink data. It also connects to the **Google Search Console API** for organic performance metrics, the **Ahrefs API** for domain authority scores, and optionally the **Google Ads API** as a fallback for SERP data.

### How does Open‑SEO validate incoming data before storage?

All incoming data is validated using **Zod schemas** located in `src/types/schemas/`. For example, keyword payloads are checked against [`keywords.ts`](https://github.com/every-app/open-seo/blob/main/keywords.ts) schemas, while audit data schemas ensure HTTP status codes and Lighthouse scores conform to expected types before being written to the PostgreSQL database via Drizzle ORM.

### Can Open‑SEO process data without consuming external API credits?

Yes. Operations involving **saved keywords**—such as tagging, note-taking, or listing previously researched keywords—are handled entirely within the internal database. The `save_keywords` and `list_saved_keywords` MCP tools in `src/server/mcp/tools/` interact only with the `saved_keywords` table, consuming no DataForSEO or Ahrefs credits.

### Where is crawl and audit data stored after processing?

Technical SEO audit data—including HTTP response classes, page titles, and Lighthouse performance scores—is stored in the `audit` schema. The crawling workflow in [`src/server/workflows/siteAuditWorkflowCrawl.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/siteAuditWorkflowCrawl.ts) persists this data through [`src/serverFunctions/audit.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/audit.ts), making it available for historical analysis and regression detection.