# How Keyword Research Works in the OpenSEO Project: A Complete Technical Guide

> Discover how OpenSEO performs keyword research using a server function pipeline. Learn about input normalization, cache checks, data provider selection, and execution modes for enriched metrics.

- Repository: [Every App/open-seo](https://github.com/every-app/open-seo)
- Tags: deep-dive
- Published: 2026-08-09

---

**OpenSEO performs keyword research through a server-function pipeline that normalizes input, checks cache, selects data providers based on market, and executes auto, manual, or Google-Ads modes before persisting enriched metrics.**

The `every-app/open-seo` repository implements a production-grade keyword research system designed for scalability and multi-market support. This article breaks down the complete architecture—from the API entry point through data persistence—so you can understand how the system handles everything from simple suggestions to complex multi-source research campaigns.

---

## Architecture Overview

The keyword research flow in OpenSEO follows a layered architecture with clear separation of concerns. Each layer handles a specific responsibility: request validation, business logic orchestration, data fetching, and persistence.

The pipeline can be summarized as:

```

Client → researchKeywords server fn → KeywordResearchService → research.ts core
    ↓
Normalization → Cache check → Provider selection → Mode execution → Persistence

```

---

## API Entry Point: The researchKeywords Server Function

All keyword research requests enter through the `researchKeywords` server function defined in [`src/serverFunctions/keywords.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/keywords.ts) at lines 26-41.

This function performs three critical tasks:

- **Request validation** – Ensures required parameters (project ID, keywords, location, language) are present and valid
- **Market resolution** – Determines the target market based on language and location codes
- **E2E fixture handling** – Returns mock data when the E2E fixture flag is set, enabling reliable testing without external API calls

For production traffic, validated requests are forwarded to `KeywordResearchService.research` rather than handled directly. This indirection allows the service layer to evolve independently of the API contract.

---

## Service Façade: KeywordResearchService

The `KeywordResearchService` in [`src/server/features/keywords/services/KeywordResearchService.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/services/KeywordResearchService.ts) (lines 14-25) acts as a thin re-export layer. It exposes the concrete `research` implementation from the research module while providing a stable interface for consumers.

This pattern—façade over implementation—enables:

- Easy swapping of research algorithms without breaking API consumers
- Consistent method signatures across the codebase
- Simplified testing through interface mocking

---

## Core Research Algorithm

The heart of OpenSEO's keyword research lives in [`src/server/features/keywords/services/research/research.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/services/research/research.ts) (lines 88-155). This orchestration layer handles deduplication, caching, provider selection, and mode-specific execution.

### Keyword Normalization and Deduplication

Before any external calls, all input keywords pass through `normalizeKeyword` in [`src/server/features/keywords/services/research/helpers.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/services/research/helpers.ts) (lines 13-15):

```typescript
// From helpers.ts
export const normalizeKeyword = (keyword: string): string =>
  keyword.trim().toLowerCase();

```

Deduplication occurs after normalization, ensuring identical keywords in different cases or with extra whitespace don't trigger redundant API requests.

### Deterministic Cache Key Generation

The `buildResearchCacheKey` utility creates a deterministic key from:

- Project identifier
- Organization ID
- Location and language codes
- Result limit
- Research mode
- Clickstream flag
- Cache version (currently 3)

Cache hits return instantly, bypassing all downstream processing. This design prioritizes response speed for repeated research queries.

### Data Provider Selection

The `getKeywordDataProvider` function chooses between two backends based on location:

| Provider | Use Case |
|----------|----------|
| **DataForSEO Labs** | Primary provider for most markets; supports related keywords, suggestions, and ideas |
| **Google Ads** | Used exclusively for Google-Ads-only markets; forced to *auto* mode with clickstream disabled |

This provider abstraction allows OpenSEO to adapt to regional API availability and customer requirements without changing client code.

---

## Research Modes: Three Execution Pathways

OpenSEO supports three mutually exclusive research modes, each optimized for different use cases.

### Google-Ads Source Mode

When Google Ads is the sole data source, `fetchGoogleAdsRows` calls the Ads-only endpoint `keywords.adsIdeas`. This mode bypasses DataForSEO entirely and returns keyword suggestions based on advertiser data.

### Auto Mode: Intelligent Multi-Source Research

The `fetchAutoRows` function implements an intelligent fallback system defined in [`src/server/features/keywords/services/research/selection.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/services/research/selection.ts) (lines 11-15):

```typescript
// Source priority order from selection.ts
export const AUTO_KEYWORD_SOURCES = [
  'related',      // Keywords semantically related to seed terms
  'suggestions',  // Search suggestions/autocomplete data
  'ideas',        // Broad keyword ideas from trend analysis
] as const;

```

The auto algorithm follows this sequence:

1. Query each source in priority order
2. After each source, check `hasSufficientCoverage` whether at least **5 non-seed keywords** have been collected
3. Stop early if coverage threshold is met; otherwise continue to next source

This **coverage-driven termination** balances comprehensiveness against API cost and latency. The threshold of 5 non-seed keywords is defined in [`selection.ts`](https://github.com/every-app/open-seo/blob/main/selection.ts) at lines 17-33.

### Manual Mode: Direct Source Access

`fetchManualRows` bypasses the auto selection logic and queries a single specified source: `related`, `suggestions`, or `ideas`. This mode gives callers precise control over data provenance.

---

## Data Fetching and Transformation

All row fetching flows through `fetchResearchRowsBySource`, which communicates with:

- DataForSEO Labs APIs: `keywords.related`, `keywords.suggestions`, `keywords.ideas`
- Google Ads API: `keywords.adsIdeas`

Raw API payloads are unstable across providers. The mapper functions in [`src/server/features/keywords/services/research/research-data.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/services/research/research-data.ts) (lines 41-59) normalize responses into the `EnrichedKeyword` type:

```typescript
// Typical EnrichedKeyword shape
{
  keyword: string;
  searchVolume: number | null;
  cpc: number | null;
  competition: number | null;
  difficulty: number | null;
  intent: 'informational' | 'navigational' | 'commercial' | 'transactional' | null;
  trend: number[] | null;
}

```

This transformation layer insulates downstream consumers from provider-specific schema changes.

---

## Persistence and Caching

Successful fetches trigger two persistence operations:

1. **Cache storage** via `setCached` in [`src/server/lib/r2-cache.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/r2-cache.ts) — enables instant retrieval for identical future requests
2. **Database upsert** via `KeywordResearchRepository.persistRows` — maintains historical keyword metrics for trend analysis

The `KeywordResearchRepository` in [`src/server/features/keywords/repositories/KeywordResearchRepository.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/repositories/KeywordResearchRepository.ts) handles deduplicated metric storage, ensuring keyword data accumulates rather than replaces previous research.

---

## Practical Implementation Examples

### Basic Auto Mode Research

Call the server function from a React component using TanStack React-Start:

```tsx
import { researchKeywords } from '@/serverFunctions/keywords';
import { useMutation } from '@tanstack/react-query';

function KeywordResearch({ projectId, keywords, location, language }) {
  const mutation = useMutation({
    mutationFn: (data) =>
      researchKeywords.mutateAsync({
        projectId,
        keywords,
        locationCode: location,
        languageCode: language,
        resultLimit: 50,
        mode: 'auto',
        clickstream: true,
      }),
  });

  const start = () => mutation.mutate();

  return (
    <div>
      <button onClick={start} disabled={mutation.isLoading}>
        Research Keywords
      </button>
      {mutation.isSuccess && (
        <ul>
          {mutation.data.rows.map((row) => (
            <li key={row.keyword}>
              {row.keyword} – {row.searchVolume ?? 'N/A'} vol, {row.intent}
            </li>
          ))}
        </ul>
      )}
    </div>
  );
}

```

### Manual Mode: Suggestions Only

Request specific source data without auto-fallback:

```ts
await researchKeywords.mutateAsync({
  projectId,
  keywords: ['photoshop tutorial'],
  locationCode: 2840,            // United States
  languageCode: 'en',
  resultLimit: 30,
  mode: 'suggestions',           // Forces "suggestions" source
  clickstream: false,
});

```

### Debug Cache Key Generation

Inspect how requests map to cache keys:

```ts
import { buildCacheKey } from '@/server/lib/r2-cache';

const key = await buildCacheKey('kw:research', {
  cacheVersion: 3,
  organizationId: 'org_123',
  projectId: 'proj_456',
  keywords: ['seo audit'],
  locationCode: 2840,
  languageCode: 'en',
  resultLimit: 20,
  mode: 'auto',
  depth: 3,
  clickstream: true,
});
console.log('Cache key →', key);

```

---

## Key Source Files Reference

| File | Responsibility |
|------|---------------|
| [`src/serverFunctions/keywords.ts`](https://github.com/every-app/open-seo/blob/main/src/serverFunctions/keywords.ts) | Public API entry point for keyword research |
| [`src/server/features/keywords/services/KeywordResearchService.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/services/KeywordResearchService.ts) | Service façade exposing research interface |
| [`src/server/features/keywords/services/research/research.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/services/research/research.ts) | Core orchestration: normalization, caching, mode dispatch |
| [`src/server/features/keywords/services/research/selection.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/services/research/selection.ts) | Auto-mode source priority and coverage logic |
| [`src/server/features/keywords/services/research/research-data.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/services/research/research-data.ts) | DataForSEO API wrappers and response mapping |
| [`src/server/features/keywords/services/research/helpers.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/services/research/helpers.ts) | Keyword and intent normalization utilities |
| [`src/server/features/keywords/repositories/KeywordResearchRepository.ts`](https://github.com/every-app/open-seo/blob/main/src/server/features/keywords/repositories/KeywordResearchRepository.ts) | Database persistence for keyword metrics |
| [`src/server/lib/r2-cache.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/r2-cache.ts) | R2-based caching infrastructure |

---

## Summary

- **Request handling**: `researchKeywords` validates input and delegates to `KeywordResearchService`
- **Normalization**: Keywords are trimmed, lower-cased, and deduplicated via `normalizeKeyword`
- **Caching**: Deterministic keys built from request parameters eliminate redundant API calls
- **Provider logic**: DataForSEO Labs for most markets; Google Ads for restricted regions
- **Auto mode**: Iterates `related → suggestions → ideas` until 5+ non-seed keywords collected
- **Data transformation**: Raw API responses mapped to stable `EnrichedKeyword` shape
- **Persistence**: Results cached to R2 and metrics upserted to repository

---

## Frequently Asked Questions

### How does OpenSEO prevent duplicate keyword research API calls?

OpenSEO uses deterministic cache key generation via `buildResearchCacheKey` in the research pipeline. Keys incorporate project ID, organization, location, language, result limit, mode, and clickstream flag. Before any external API call, the system checks [`src/server/lib/r2-cache.ts`](https://github.com/every-app/open-seo/blob/main/src/server/lib/r2-cache.ts) for existing results and returns cached data instantly when available.

### What determines whether OpenSEO uses DataForSEO Labs or Google Ads?

The `getKeywordDataProvider` function selects based on market configuration. Most locations use DataForSEO Labs with full mode support. Google-Ads-only markets force the provider to Google Ads, disable clickstream data, and restrict mode to `auto`. This adaptation ensures compliance with regional data availability.

### Can I control which keyword sources OpenSEO queries?

Yes. Set `mode` to `related`, `suggestions`, or `ideas` for direct single-source queries. Use `mode: 'auto'` for intelligent multi-source research that stops when coverage exceeds 5 non-seed keywords. Set `mode: 'google-ads'` (implicit in restricted markets) for advertiser-specific keyword ideas only.

### Where does OpenSEO store keyword research results?

Results persist in two locations: R2 object storage via `setCached` for fast retrieval of identical queries, and the database via `KeywordResearchRepository.persistRows` for historical analysis. The dual persistence strategy optimizes both performance and long-term data access.