# How OpenSEO Protects Against SSRF Vulnerabilities During Site Audits

> Learn how OpenSEO safeguards against SSRF vulnerabilities during site audits. Discover its URL policy module, scheme restrictions, and blocked host list for enhanced security.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: security
- Published: 2026-08-30

---

**OpenSEO prevents SSRF attacks by validating all URLs through a dedicated URL policy module that enforces scheme restrictions, maintains a blocked host list, and performs DNS-over-HTTPS resolution to detect private IP addresses before any request leaves the crawler.**

OpenSEO is an open-source SEO audit platform that executes its site-audit crawler within a Cloudflare Workers environment. Because users supply arbitrary start URLs and the crawler follows links discovered during execution, the system must rigorously defend against Server-Side Request Forgery (SSRF) attacks that could probe internal networks or metadata services. According to the OpenSEO source code, this protection is centralized in [`src/server/lib/audit/url-policy.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/url-policy.ts) and applied at three distinct stages: initial URL validation, per-link crawling decisions, and redirect chain re-validation.

## Start-URL Validation with normalizeAndValidateStartUrl

Before any crawl begins, OpenSEO normalizes and vets the user-supplied start URL through the asynchronous function `normalizeAndValidateStartUrl` (lines 9‑38 in [`src/server/lib/audit/url-policy.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/audit/url-policy.ts)). This function acts as the first line of defense, ensuring that the audit cannot be seeded with a malicious internal address.

### Scheme and Hostname Filtering

The validator first enforces that the URL scheme is strictly `http` or `https`, rejecting any alternative protocols that could facilitate code execution or file system access. It then checks the hostname against a **blocked host set** defined at the top of the module (lines 3‑9), which includes entries like `localhost`, GCE metadata IP addresses (`169.254.169.254`), and other common internal endpoints.

### DNS Resolution and Private IP Blocking

To prevent DNS rebinding attacks where a hostname initially resolves to a public IP but later switches to a private one, OpenSEO performs a **DNS-over-HTTPS (DoH)** lookup using `resolveAddressRecords` (lines 42‑64) against Cloudflare’s resolver. The resolved addresses are passed to `hostnameResolvesToBlockedAddress`, which utilizes `isPrivateIpv4` and `isPrivateIpv6` (lines 35‑55 and 66‑84) to detect addresses within restricted ranges—including mapped IPv6 addresses—before the crawler fetches any content.

## Per-Link SSRF Checks During Crawling

While the audit progresses, the crawler discovers links, sitemap entries, and redirect targets that require lightweight, synchronous validation to maintain performance.

### The isCrawlableUrl Filter

The exported function `isCrawlableUrl` (lines 96‑107) parses each discovered URL and immediately rejects non-HTTP schemes. It delegates hostname validation to `isBlockedHost`, which ensures the target does not appear in the forbidden list and is not a literal private IP address. This check executes synchronously to avoid the latency of DNS queries during high-volume crawling.

### Integration in the Crawl Loop

Within the crawl workflow defined in [`src/server/workflows/siteAuditWorkflowCrawl.ts`](https://github.com/every-app/open-seo/blob/main/src/server/workflows/siteAuditWorkflowCrawl.ts), the function `shouldQueueCrawlLink` (lines 70‑78) invokes `isCrawlableUrl` as a prerequisite for queueing any newly discovered link. This ensures that **every URL**—whether from an anchor tag, redirect, or sitemap—passes SSRF scrutiny before the fetch worker receives it.

## Redirect Chain Re-validation

SSRF attackers often attempt to bypass filters by using a public URL that redirects to an internal address. OpenSEO neutralizes this vector through hop-by-hop validation.

### resolveStartUrlRedirects and Hop-by-Hop Security

The function `resolveStartUrlRedirects` (lines 45‑85) follows up to five redirect hops, calling `normalizeAndValidateStartUrl` on every intermediate URL in the chain. This guarantees that no redirect can smuggle the audit into a blocked address space, even if the initial URL appeared legitimate. The architecture is documented in the design specification at [`specs/0009-site-audit-crawl-architecture.md`](https://github.com/every-app/open-seo/blob/main/specs/0009-site-audit-crawl-architecture.md) (line 93).

### Code Examples

The following patterns demonstrate how OpenSEO integrates these protections into its audit workflow:

```typescript
// Validate the audit start URL when a user creates an audit
import { normalizeAndValidateStartUrl } from "@/server/lib/audit/url-policy";

async function startAudit(inputUrl: string) {
  const safeUrl = await normalizeAndValidateStartUrl(inputUrl);
  // safeUrl is guaranteed to be HTTP(S) and not point to a blocked host
}

```

```typescript
// Guard each discovered link before queuing it for crawling
import { isCrawlableUrl } from "@/server/lib/audit/url-policy";

function shouldQueueCrawlLink(
  link: string,
  origin: string,
  robots: RobotsResult,
) {
  // Only same-origin, allowed by robots.txt, and passes SSRF check
  return isSameOrigin(link, origin) && isCrawlableUrl(link) && robots.isAllowed(link);
}

```

```typescript
// Follow redirects with per-hop SSRF revalidation
import { resolveStartUrlRedirects } from "@/server/lib/audit/url-policy";

const finalUrl = await resolveStartUrlRedirects(startUrl);
// finalUrl has survived all redirect-hop checks

```

## Summary

- **Start-URL validation** in `normalizeAndValidateStartUrl` blocks non-HTTP schemes, forbidden hostnames, and private IP ranges using DoH resolution before crawling begins.
- **Per-link filtering** via `isCrawlableUrl` ensures every discovered URL passes synchronous SSRF checks in the crawl loop ([`siteAuditWorkflowCrawl.ts`](https://github.com/every-app/open-seo/blob/main/siteAuditWorkflowCrawl.ts)).
- **Redirect re-validation** in `resolveStartUrlRedirects` applies the same strict rules to every hop in a redirect chain (up to five levels deep).
- **Blocked host definitions** explicitly blacklist localhost, cloud metadata endpoints, and private IPv4/IPv6 ranges to prevent internal network probing.

## Frequently Asked Questions

### What is SSRF and why does OpenSEO specifically need protection against it?

Server-Side Request Forgery (SSRF) is a vulnerability where an attacker tricks a server into making requests to unintended destinations, often allowing access to internal services or cloud metadata APIs. OpenSEO specifically requires protection because it fetches arbitrary user-supplied URLs during site audits; without safeguards, an attacker could use the crawler to scan internal networks or retrieve sensitive instance metadata from the Cloudflare Workers environment.

### How does OpenSEO prevent DNS rebinding attacks during audits?

OpenSEO prevents DNS rebinding by performing a **DNS-over-HTTPS lookup** via `resolveAddressRecords` when validating the start URL. This resolves the hostname to concrete IP addresses before any HTTP request is made, allowing `isPrivateIpv4` and `isPrivateIpv6` to detect if the resolution points to restricted ranges (lines 35‑84 in [`url-policy.ts`](https://github.com/every-app/open-seo/blob/main/url-policy.ts)). By validating the resolved IP rather than just the hostname string, the system blocks hostnames that might dynamically switch to internal addresses after validation.

### Why does OpenSEO use synchronous checks for discovered links but asynchronous DNS for start URLs?

OpenSEO uses asynchronous DNS resolution (DoH) for start URLs because this is a one-time cost per audit that allows deep validation of the initial entry point. For discovered links, it uses the synchronous `isCrawlableUrl` check to maintain crawling performance, examining only the hostname string against the blocked list and rejecting literal private IP addresses without performing a network lookup. This design balances security with the high throughput required for large-scale site audits.

### What happens if a redirect during crawling points to a private IP address?

If a redirect points to a private IP address, the `resolveStartUrlRedirects` function (lines 45‑85 in [`url-policy.ts`](https://github.com/every-app/open-seo/blob/main/url-policy.ts)) detects the violation when it calls `normalizeAndValidateStartUrl` on each redirect hop. The function throws an error and terminates the redirect following process, preventing the crawler from ever reaching the internal address. This per-hop re-validation ensures that redirect chains cannot be used to bypass initial SSRF filters.