# How the OpenSEO Keyword Clustering Algorithm Works: Cosine Similarity and SERP Intent

> Discover how OpenSEO's keyword clustering algorithm uses cosine similarity and SERP intent to group keywords for better content strategy. Learn about Gemma 300M and Vectorize.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: deep-dive
- Published: 2026-07-30

---

**OpenSEO's keyword clustering algorithm combines cosine similarity of text embeddings with SERP intent validation to group keywords into coherent content clusters, using Cloudflare Vectorize and embedding models like Google's Gemma 300M.**

OpenSEO (from the [every-app/open-seo](https://github.com/every-app/open-seo) repository) implements a sophisticated keyword clustering algorithm that transforms scattered keyword lists into strategic page-level groups. The algorithm, defined in [`.agents/skills/keyword-clustering/SKILL.md`](https://github.com/every-app/open-seo/blob/main/.agents/skills/keyword-clustering/SKILL.md), executes a three-step pipeline that prioritizes semantic similarity while enforcing strict search intent guardrails to prevent purely lexical clustering mistakes.

## The Three-Step Keyword Clustering Pipeline

The clustering workflow processes keywords through distinct phases that combine vector mathematics with real-world search validation data.

### Step 1: Embedding Generation with Cloudflare Models

Each keyword phrase is converted into a high-dimensional vector using **Cloudflare CF Google EmbeddingGemma-300M**, a cloud-hosted embedding model that captures semantic meaning beyond simple term matching. These vectors encode contextual relationships and thematic similarities between queries, creating the mathematical foundation for clustering.

### Step 2: Cosine Similarity Calculation in Vectorize

The generated vectors are stored in **Cloudflare Vectorize**, a vector-search service configured to use specific distance metrics. According to [`worker-configuration.d.ts`](https://github.com/every-app/open-seo/blob/main/worker-configuration.d.ts) at lines 10597-10620, the `VectorizeDistanceMetric` type supports three options: `"euclidean"`, `"cosine"`, and `"dot-product"`. The keyword-clustering skill explicitly selects **cosine similarity**, which measures the angle between two vectors rather than their absolute distance. This metric is optimal for textual semantics because it normalizes for vector magnitude and focuses on directional alignment, effectively grouping terms that point in similar semantic directions regardless of their frequency or length.

### Step 3: SERP Intent Validation and Page Overlap

Provisional clusters undergo refinement through **SERP overlap analysis** and intent checking. The algorithm makes lightweight `get_serp_results` calls for borderline terms, validating whether keywords share the same ranking pages or SERP features. The SKILL.md guardrails explicitly state: *"Do not rely on lexical similarity alone. SERP intent wins."* Keywords with divergent search intents—for example, commercial versus informational queries—are forced into separate clusters regardless of their embedding similarity, ensuring the final output aligns with actual user behavior.

## Similarity Metrics: Technical Implementation Details

While Cloudflare Vectorize supports three distance metrics, OpenSEO's implementation strategically selects cosine similarity for specific technical advantages in text analysis:

- **Cosine similarity**: Calculates the cosine of the angle between vectors (range -1 to 1). This metric excels with high-dimensional text embeddings where vector magnitude varies but directional alignment indicates semantic proximity.
- **Euclidean distance**: Measures straight-line distance between points. The algorithm avoids this for keyword clustering because absolute distances in high-dimensional embedding spaces can obscure semantic relationships.
- **Dot product**: Measures vector projection. This metric can be influenced by vector length, making it less reliable than cosine for normalized semantic comparison.

The configuration in [`worker-configuration.d.ts`](https://github.com/every-app/open-seo/blob/main/worker-configuration.d.ts) confirms the clustering operation explicitly sets `VectorizeDistanceMetric` to `"cosine"` to maintain consistent semantic grouping behavior.

## Implementing Keyword Clustering in Your Workflow

You can invoke the clustering algorithm through the UI or programmatically via the skill API.

Trigger the skill from the React UI using the `RunSkillCallout` component:

```tsx
// Trigger the clustering skill from the UI (run-skill callout)
<RunSkillCallout command="/keyword-clustering" />

```

Execute the algorithm programmatically from server-side routes using the `fetchSkill` function:

```ts
// Programmatic call from a server-side route (e.g., /api/cluster)
import { fetchSkill } from '@/lib/ai'

await fetchSkill({
  name: 'keyword-clustering',
  inputs: {
    projectId: '123',
    keywordList: ['affordable semrush alternatives', 'cheap semrush', 'semrush pricing'],
    // optional: existingPages: [{ url: '/pricing' }, …]
  },
})

```

The algorithm returns structured cluster data formatted as a markdown table:

```ts
// Example of the returned cluster table (Markdown format)
| Cluster | Primary keyword | Secondary keywords            | Intent      | Target page          | Priority | Notes                         |
| ------- | --------------- | ----------------------------- | ----------- | -------------------- | -------- | ----------------------------- |
| 1       | affordable semrush alternatives | cheap semrush, semrush pricing | Commercial  | /pricing/semrush    | High     | Strong SERP overlap, same intent |
| 2       | semrush review  | semrush pros and cons, semrush vs ahrefs | Informational | /blog/semrush-review | Medium   | Distinct SERP, separate page   |

```

## Configuration Files and Skill Guardrails

The clustering behavior is governed by specific configuration files:

- [`.agents/skills/keyword-clustering/SKILL.md`](https://github.com/every-app/open-seo/blob/main/.agents/skills/keyword-clustering/SKILL.md) defines the workflow, guardrails, and required inputs
- [`src/routes/_app/ai.tsx`](https://github.com/every-app/open-seo/blob/main/src/routes/_app/ai.tsx) registers the `"keyword-clustering"` command for UI invocation
- [`src/server/mcp/tools/saved-keywords-tools.test.ts`](https://github.com/every-app/open-seo/blob/main/src/server/mcp/tools/saved-keywords-tools.test.ts) demonstrates how clustering tags (e.g., `cluster: affordable semrush alternatives`) are persisted to saved keyword records

The system maintains cluster relationships through persistent tagging, ensuring mappings remain accessible across analysis sessions.

## Summary

- OpenSEO's keyword clustering algorithm uses **cosine similarity** of text embeddings to measure semantic relationships between keywords.
- The three-step pipeline includes: embedding generation via Google Gemma 300M, vector similarity calculation in Cloudflare Vectorize, and **SERP intent validation** that overrides purely semantic similarity.
- The algorithm explicitly avoids Euclidean and dot-product metrics in favor of cosine similarity for accurate text semantic comparison.
- Implementation supports both UI-triggered execution via `RunSkillCallout` and programmatic calls via `fetchSkill`.
- Output includes structured cluster tables with primary keywords, secondary keywords, inferred intent, suggested target pages, and priority levels.

## Frequently Asked Questions

### What similarity metric does OpenSEO use for keyword clustering?

OpenSEO uses **cosine similarity** as configured in the Cloudflare Vectorize service. This metric, defined in [`worker-configuration.d.ts`](https://github.com/every-app/open-seo/blob/main/worker-configuration.d.ts) as part of the `VectorizeDistanceMetric` enum (`"euclidean" | "cosine" | "dot-product"`), measures the angle between text embedding vectors to capture semantic relationships regardless of vector magnitude. The keyword-clustering skill explicitly selects this metric for all clustering operations.

### How does the algorithm handle keywords with similar meanings but different search intents?

The algorithm implements **SERP intent validation** as a critical guardrail. After initial vector-based grouping, the system checks actual search engine results for borderline terms. Keywords appearing on different SERPs or serving distinct user intents—such as transactional versus informational queries—are automatically split into separate clusters. This "SERP intent wins" rule ensures clusters align with real-world search behavior rather than purely mathematical similarity.

### What embedding model powers the OpenSEO clustering algorithm?

The clustering algorithm utilizes **Cloudflare CF Google EmbeddingGemma-300M**, a cloud-hosted embedding model that converts keyword phrases into high-dimensional vectors using 300 million parameters. These embeddings capture nuanced semantic relationships between terms, enabling the cosine similarity calculations that drive the initial clustering phase before SERP validation refines the groups.

### Can I customize the distance metric or clustering thresholds in OpenSEO?

While the underlying [`worker-configuration.d.ts`](https://github.com/every-app/open-seo/blob/main/worker-configuration.d.ts) supports three distance metrics (`"euclidean"`, `"cosine"`, and `"dot-product"`), the keyword-clustering skill in [`.agents/skills/keyword-clustering/SKILL.md`](https://github.com/every-app/open-seo/blob/main/.agents/skills/keyword-clustering/SKILL.md) is hardcoded to use cosine similarity. Customizing the metric would require modifying the skill configuration files, as the current implementation enforces cosine similarity to maintain consistency with the SERP intent validation logic and semantic clustering objectives.