# What Kind of Data Can Open-SEO Extract? A Complete Technical Breakdown

> Discover what kind of data Open-SEO extracts, from high-level site snapshots like URLs and titles to granular SEO audit data including meta tags, links, and structured data. A complete technical breakdown.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: deep-dive
- Published: 2026-08-06

---

**Open-SEO extracts both high-level site snapshots (URLs, titles, and plain text content) and granular SEO audit data (meta tags, headings, links, images, structured data, and hreflang tags) through its modular TypeScript functions.**

The `every-app/open-seo` repository provides a TypeScript-based toolkit for programmatic website analysis. Understanding exactly what kind of data Open-SEO can extract helps developers build comprehensive audit pipelines and content processing workflows.

## Site-Wide Content Discovery and Reading

### Automated URL Discovery via Sitemaps

The `discoverSiteUrls` function in [`src/server/lib/scrape.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/scrape.ts) performs automated site reconnaissance. It extracts the **homepage URL** and parses XML sitemaps to return same-origin HTML pages, while [`src/server/lib/audit/url-policy.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/url-policy.ts) applies SSRF guards to validate and sanitize URLs before any network request occurs.

### Page Content Extraction

The `readPages` function fetches up to five pages (configurable) and processes each through `scrapePage`. This pipeline uses `fetchText`, `extractTitle`, and `htmlToText` to return a `ScrapedPage` object containing:

- **Page URL**
- **Page title** (from the HTML `<title>` tag)
- **Plain-text content** (cleaned HTML stripped of tags)

## Comprehensive SEO Analysis with analyzeHtml

The `analyzeHtml` function in [`src/server/lib/audit/page-analyzer.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/page-analyzer.ts) uses **cheerio** to parse HTML and extract detailed SEO signals.

### Meta Tags and Social Graph Data

The analyzer captures critical meta information:

- **Title** tag content
- **Meta description**
- **Canonical URL**
- **Robots meta tag** directives
- **Open Graph tags** (title, description, image)

### Document Structure and Media Assets

Content structure is evaluated through:

- **All headings** (`h1` through `h6`) and their hierarchical order
- **Word count** of visible body text
- **Images** with their `src` URLs and `alt` text attributes

### Link Profile and Navigation Analysis

The system catalogs every anchor tag with:

- **Destination URL** and anchor text
- **Internal/external** classification flags
- **No-follow** attribute detection

### Technical SEO Signals

Advanced extraction includes:

- **Structured data** presence (specifically `application/ld+json` script tags)
- **Hreflang tags** for internationalization signals

## Practical Implementation Examples

The following TypeScript examples demonstrate how to leverage Open-SEO's extraction capabilities:

```typescript
// 1️⃣ Discover URLs for a domain (homepage + sitemap)
import { discoverSiteUrls } from "./src/server/lib/scrape";

const { urls, blocked } = await discoverSiteUrls("https://example.com", 10);
if (blocked) console.log("Site unreachable or blocked");

```

```typescript
// 2️⃣ Read up to 5 pages (title + plain text)
import { readPages } from "./src/server/lib/scrape";

const { pages, blocked: readBlocked } = await readPages(urls);
pages.forEach(p => {
  console.log(`🗒️ ${p.title ?? "No title"} → ${p.text.slice(0, 200)}…`);
});

```

```typescript
// 3️⃣ Run a full SEO analysis on a page’s HTML
import { analyzeHtml } from "./src/server/lib/audit/page-analyzer";

const html = await fetch(pages[0].url).then(r => r.text());
const analysis = analyzeHtml(html, pages[0].url, 200, 120);
console.log("SEO title:", analysis.title);
console.log("Meta description:", analysis.metaDescription);
console.log("Headings:", analysis.h1s);
console.log("Word count:", analysis.wordCount);
console.log("Links (first 5):", analysis.links.slice(0, 5));

```

## Key Source Files and Architecture

Understanding the codebase structure clarifies how data extraction operates:

- **[`src/server/lib/scrape.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/scrape.ts)**: Handles site discovery, bounded fetching, and title/plain-text extraction.
- **[`src/server/lib/audit/page-analyzer.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/page-analyzer.ts)**: Contains the cheerio-based SEO audit logic.
- **[`src/server/lib/audit/types.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/types.ts)**: Defines TypeScript interfaces for `PageAnalysis` results.
- **[`src/server/lib/audit/url-utils.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/url-utils.ts)**: Provides URL normalization helpers used throughout the analyzer.

## Summary

- **Open-SEO extracts two primary data layers**: high-level content snapshots and detailed SEO audit signals.
- **Content reading** functions in [`scrape.ts`](https://github.com/every-app/open-seo/blob/main/scrape.ts) provide URLs, titles, and cleaned plain text from multiple pages.
- **SEO analysis** via `analyzeHtml` captures meta tags, headings (h1-h6), images with alt text, links with internal/external flags, word counts, structured data, and hreflang tags.
- **Discovery features** automatically parse sitemaps and validate URLs against SSRF policies before fetching.

## Frequently Asked Questions

### Does Open-SEO extract JavaScript-rendered content?

No, the current implementation in [`src/server/lib/scrape.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/scrape.ts) uses standard HTTP fetching and cheerio-based HTML parsing, which processes the initial server-rendered markup only. It does not execute JavaScript or wait for client-side hydration.

### How many pages can Open-SEO process in a single run?

The `readPages` function defaults to processing five pages, though this limit is configurable. The `discoverSiteUrls` function accepts a maximum URL parameter to bound the discovery phase, preventing unbounded crawling.

### What format does the structured data extraction return?

The `analyzeHtml` function detects the presence of `application/ld+json` script tags but currently reports their presence as part of the technical SEO audit. The specific parsing of JSON-LD content into structured objects depends on the implementation details in [`src/server/lib/audit/page-analyzer.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/page-analyzer.ts).

### Can Open-SEO differentiate between internal and external links?

Yes, the link extraction logic in [`src/server/lib/audit/page-analyzer.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/page-analyzer.ts) classifies each link with an **internal/external flag** based on the URL origin, and also detects **no-follow** attributes, providing complete link profile data for audit reports.