How Keyword Research Works in the OpenSEO Project: A Complete Technical Guide

OpenSEO performs keyword research through a server-function pipeline that normalizes input, checks cache, selects data providers based on market, and executes auto, manual, or Google-Ads modes before persisting enriched metrics.

The every-app/open-seo repository implements a production-grade keyword research system designed for scalability and multi-market support. This article breaks down the complete architecture—from the API entry point through data persistence—so you can understand how the system handles everything from simple suggestions to complex multi-source research campaigns.

---

Architecture Overview

The keyword research flow in OpenSEO follows a layered architecture with clear separation of concerns. Each layer handles a specific responsibility: request validation, business logic orchestration, data fetching, and persistence.

The pipeline can be summarized as:

Client → researchKeywords server fn → KeywordResearchService → research.ts core
    ↓
Normalization → Cache check → Provider selection → Mode execution → Persistence

---

API Entry Point: The researchKeywords Server Function

All keyword research requests enter through the researchKeywords server function defined in src/serverFunctions/keywords.ts at lines 26-41.

This function performs three critical tasks:

• Request validation – Ensures required parameters (project ID, keywords, location, language) are present and valid
• Market resolution – Determines the target market based on language and location codes
• E2E fixture handling – Returns mock data when the E2E fixture flag is set, enabling reliable testing without external API calls

For production traffic, validated requests are forwarded to KeywordResearchService.research rather than handled directly. This indirection allows the service layer to evolve independently of the API contract.

---

Service Façade: KeywordResearchService

The KeywordResearchService in src/server/features/keywords/services/KeywordResearchService.ts (lines 14-25) acts as a thin re-export layer. It exposes the concrete research implementation from the research module while providing a stable interface for consumers.

This pattern—façade over implementation—enables:

• Easy swapping of research algorithms without breaking API consumers
• Consistent method signatures across the codebase
• Simplified testing through interface mocking

---

Core Research Algorithm

The heart of OpenSEO's keyword research lives in src/server/features/keywords/services/research/research.ts (lines 88-155). This orchestration layer handles deduplication, caching, provider selection, and mode-specific execution.

Keyword Normalization and Deduplication

Before any external calls, all input keywords pass through normalizeKeyword in src/server/features/keywords/services/research/helpers.ts (lines 13-15):

// From helpers.ts
export const normalizeKeyword = (keyword: string): string =>
  keyword.trim().toLowerCase();

Deduplication occurs after normalization, ensuring identical keywords in different cases or with extra whitespace don't trigger redundant API requests.

Deterministic Cache Key Generation

The buildResearchCacheKey utility creates a deterministic key from:

• Project identifier
• Organization ID
• Location and language codes
• Result limit
• Research mode
• Clickstream flag
• Cache version (currently 3)

Cache hits return instantly, bypassing all downstream processing. This design prioritizes response speed for repeated research queries.

Data Provider Selection

The getKeywordDataProvider function chooses between two backends based on location:

Provider	Use Case
DataForSEO Labs	Primary provider for most markets; supports related keywords, suggestions, and ideas
Google Ads	Used exclusively for Google-Ads-only markets; forced to auto mode with clickstream disabled

This provider abstraction allows OpenSEO to adapt to regional API availability and customer requirements without changing client code.

---

Research Modes: Three Execution Pathways

OpenSEO supports three mutually exclusive research modes, each optimized for different use cases.

Google-Ads Source Mode

When Google Ads is the sole data source, fetchGoogleAdsRows calls the Ads-only endpoint keywords.adsIdeas. This mode bypasses DataForSEO entirely and returns keyword suggestions based on advertiser data.

Auto Mode: Intelligent Multi-Source Research

The fetchAutoRows function implements an intelligent fallback system defined in src/server/features/keywords/services/research/selection.ts (lines 11-15):

// Source priority order from selection.ts
export const AUTO_KEYWORD_SOURCES = [
  'related',      // Keywords semantically related to seed terms
  'suggestions',  // Search suggestions/autocomplete data
  'ideas',        // Broad keyword ideas from trend analysis
] as const;

The auto algorithm follows this sequence:

1. Query each source in priority order
2. After each source, check hasSufficientCoverage whether at least 5 non-seed keywords have been collected
3. Stop early if coverage threshold is met; otherwise continue to next source

This coverage-driven termination balances comprehensiveness against API cost and latency. The threshold of 5 non-seed keywords is defined in selection.ts at lines 17-33.

Manual Mode: Direct Source Access

fetchManualRows bypasses the auto selection logic and queries a single specified source: related, suggestions, or ideas. This mode gives callers precise control over data provenance.

---

Data Fetching and Transformation

All row fetching flows through fetchResearchRowsBySource, which communicates with:

• DataForSEO Labs APIs: keywords.related, keywords.suggestions, keywords.ideas
• Google Ads API: keywords.adsIdeas

Raw API payloads are unstable across providers. The mapper functions in src/server/features/keywords/services/research/research-data.ts (lines 41-59) normalize responses into the EnrichedKeyword type:

// Typical EnrichedKeyword shape
{
  keyword: string;
  searchVolume: number | null;
  cpc: number | null;
  competition: number | null;
  difficulty: number | null;
  intent: 'informational' | 'navigational' | 'commercial' | 'transactional' | null;
  trend: number[] | null;
}

This transformation layer insulates downstream consumers from provider-specific schema changes.

---

Persistence and Caching

Successful fetches trigger two persistence operations:

1. Cache storage via setCached in src/server/lib/r2-cache.ts — enables instant retrieval for identical future requests
2. Database upsert via KeywordResearchRepository.persistRows — maintains historical keyword metrics for trend analysis

The KeywordResearchRepository in src/server/features/keywords/repositories/KeywordResearchRepository.ts handles deduplicated metric storage, ensuring keyword data accumulates rather than replaces previous research.

---

Practical Implementation Examples

Basic Auto Mode Research

Call the server function from a React component using TanStack React-Start:

import { researchKeywords } from '@/serverFunctions/keywords';
import { useMutation } from '@tanstack/react-query';

function KeywordResearch({ projectId, keywords, location, language }) {
  const mutation = useMutation({
    mutationFn: (data) =>
      researchKeywords.mutateAsync({
        projectId,
        keywords,
        locationCode: location,
        languageCode: language,
        resultLimit: 50,
        mode: 'auto',
        clickstream: true,
      }),
  });

  const start = () => mutation.mutate();

  return (
    <div>
      <button onClick={start} disabled={mutation.isLoading}>
        Research Keywords
      </button>
      {mutation.isSuccess && (
        <ul>
          {mutation.data.rows.map((row) => (
            <li key={row.keyword}>
              {row.keyword} – {row.searchVolume ?? 'N/A'} vol, {row.intent}
            </li>
          ))}
        </ul>
      )}
    </div>
  );
}

Manual Mode: Suggestions Only

Request specific source data without auto-fallback:

await researchKeywords.mutateAsync({
  projectId,
  keywords: ['photoshop tutorial'],
  locationCode: 2840,            // United States
  languageCode: 'en',
  resultLimit: 30,
  mode: 'suggestions',           // Forces "suggestions" source
  clickstream: false,
});

Debug Cache Key Generation

Inspect how requests map to cache keys:

import { buildCacheKey } from '@/server/lib/r2-cache';

const key = await buildCacheKey('kw:research', {
  cacheVersion: 3,
  organizationId: 'org_123',
  projectId: 'proj_456',
  keywords: ['seo audit'],
  locationCode: 2840,
  languageCode: 'en',
  resultLimit: 20,
  mode: 'auto',
  depth: 3,
  clickstream: true,
});
console.log('Cache key →', key);

---

Key Source Files Reference

File	Responsibility
src/serverFunctions/keywords.ts	Public API entry point for keyword research
src/server/features/keywords/services/KeywordResearchService.ts	Service façade exposing research interface
src/server/features/keywords/services/research/research.ts	Core orchestration: normalization, caching, mode dispatch
src/server/features/keywords/services/research/selection.ts	Auto-mode source priority and coverage logic
src/server/features/keywords/services/research/research-data.ts	DataForSEO API wrappers and response mapping
src/server/features/keywords/services/research/helpers.ts	Keyword and intent normalization utilities
src/server/features/keywords/repositories/KeywordResearchRepository.ts	Database persistence for keyword metrics
src/server/lib/r2-cache.ts	R2-based caching infrastructure

---

Summary

• Request handling: researchKeywords validates input and delegates to KeywordResearchService
• Normalization: Keywords are trimmed, lower-cased, and deduplicated via normalizeKeyword
• Caching: Deterministic keys built from request parameters eliminate redundant API calls
• Provider logic: DataForSEO Labs for most markets; Google Ads for restricted regions
• Auto mode: Iterates related → suggestions → ideas until 5+ non-seed keywords collected
• Data transformation: Raw API responses mapped to stable EnrichedKeyword shape
• Persistence: Results cached to R2 and metrics upserted to repository

---

Frequently Asked Questions

How does OpenSEO prevent duplicate keyword research API calls?

OpenSEO uses deterministic cache key generation via buildResearchCacheKey in the research pipeline. Keys incorporate project ID, organization, location, language, result limit, mode, and clickstream flag. Before any external API call, the system checks src/server/lib/r2-cache.ts for existing results and returns cached data instantly when available.

What determines whether OpenSEO uses DataForSEO Labs or Google Ads?

The getKeywordDataProvider function selects based on market configuration. Most locations use DataForSEO Labs with full mode support. Google-Ads-only markets force the provider to Google Ads, disable clickstream data, and restrict mode to auto. This adaptation ensures compliance with regional data availability.

Can I control which keyword sources OpenSEO queries?

Yes. Set mode to related, suggestions, or ideas for direct single-source queries. Use mode: 'auto' for intelligent multi-source research that stops when coverage exceeds 5 non-seed keywords. Set mode: 'google-ads' (implicit in restricted markets) for advertiser-specific keyword ideas only.

Where does OpenSEO store keyword research results?

Results persist in two locations: R2 object storage via setCached for fast retrieval of identical queries, and the database via KeywordResearchRepository.persistRows for historical analysis. The dual persistence strategy optimizes both performance and long-term data access.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}