How Keyword Research Works in OpenSEO: Server-Side Architecture Explained

OpenSEO performs keyword research through a server function that normalizes input keywords, checks distributed caches, selects data providers based on market location, and executes multi-source fetching strategies—auto, manual, or Google Ads—before persisting enriched metrics.

Keyword research in OpenSEO operates as a sophisticated server-side pipeline within the every-app/open-seo repository. The system aggregates data from DataForSEO Labs and Google Ads APIs while implementing intelligent caching and deduplication strategies. This architecture delivers comprehensive keyword metrics—including search volume, CPC, competition scores, and intent classification—through a type-safe TypeScript implementation.

The Research Pipeline Architecture

The keyword research flow follows a deterministic pipeline from client request to persisted results. In src/serverFunctions/keywords.ts, the researchKeywords server function validates incoming requests and resolves market parameters (language and location). If the E2E fixture flag is active, the function returns mock data immediately; otherwise, it delegates to KeywordResearchService.research.

The service façade in src/server/features/keywords/services/KeywordResearchService.ts (lines 14-25) re-exports the concrete implementation from the research module. The core algorithm in src/server/features/keywords/services/research/research.ts orchestrates normalization, caching, provider selection, and multi-mode data fetching before persisting results through KeywordResearchRepository and the R2 cache layer.

Entry Point and Request Validation

The researchKeywords Server Function

The primary entry point resides in src/serverFunctions/keywords.ts (lines 26-41). This server function accepts project context, seed keywords, location codes, and research mode parameters. It performs request validation, resolves the target market (language/location), and checks for E2E fixture flags to determine whether to return mock data or proceed to the live research pipeline.

Upon validation, the function forwards the request to KeywordResearchService.research, which acts as the main orchestration point for the entire keyword research in OpenSEO workflow.

Core Research Orchestration

KeywordResearchService Facade

The KeywordResearchService in src/server/features/keywords/services/KeywordResearchService.ts implements a thin façade pattern. It exposes the research method that simply re-exports the concrete implementation from src/server/features/keywords/services/research/research.ts. This abstraction allows the underlying research logic to evolve independently while maintaining a stable contract for server function consumers.

The research.ts Implementation

The orchestration logic in src/server/features/keywords/services/research/research.ts (lines 55-88) implements the core workflow:

1. Deduplication and normalization – All supplied keywords are trimmed and lower-cased via normalizeKeyword defined in src/server/features/keywords/services/research/helpers.ts (lines 13-15).
2. Cache key generation – The system builds a deterministic cache key using buildResearchCacheKey based on project, location, language, result limit, mode, and depth parameters.
3. Provider selection – getKeywordDataProvider chooses between DataForSEO Labs and Google Ads based on the target location. For Google-Ads-only markets, the request is forced to auto mode with clickstream disabled.
4. Mode execution – The system branches into three pathways: Google Ads source, auto mode, or manual mode.

Data Processing and Normalization

Keyword Normalization

Before processing, all seed keywords undergo normalization through normalizeKeyword in src/server/features/keywords/services/research/helpers.ts. This utility trims whitespace and converts keywords to lowercase to ensure consistent cache keys and eliminate duplicate variations of the same term. The companion normalizeIntent function standardizes intent classifications across different data providers.

Cache Key Generation

The buildResearchCacheKey function generates deterministic keys incorporating organization ID, project ID, normalized keywords, location code (e.g., 2840 for United States), language code, result limit, mode, depth, and clickstream flags. Cached results are retrieved instantly if present, bypassing expensive API calls to DataForSEO or Google Ads.

Multi-Source Data Acquisition

Provider Selection Strategy

The getKeywordDataProvider function determines whether to route requests through DataForSEO Labs or the Google Ads API based on market availability. For locations where only Google Ads data is available, the system automatically forces auto mode and disables clickstream data to ensure compliance with provider constraints.

Auto Mode and Source Prioritization

The auto mode implementation in src/server/features/keywords/services/research/selection.ts (lines 11-15) defines the AUTO_KEYWORD_SOURCES array with a specific priority order:

• related (related keywords)
• suggestions (search suggestions)
• ideas (keyword ideas)

The algorithm in fetchAutoRows iterates through this ordered list, checking after each source whether hasSufficientCoverage returns true. The coverage threshold is set to 5 non-seed keywords. Once the system collects five or more unique keywords from the sources, it stops fetching to optimize API quota usage.

Manual and Google Ads Modes

Manual mode allows direct querying of a single source via fetchManualRows, bypassing the auto-selection logic. Valid mode values include 'related', 'suggestions', or 'ideas'.

Google Ads mode executes fetchGoogleAdsRows, which calls the Ads-only endpoint keywords.adsIdeas without querying DataForSEO Labs. This mode is ideal for users requiring advertising metrics rather than organic search data.

The underlying fetchResearchRowsBySource function communicates with DataForSEO Labs endpoints (keywords.related, keywords.suggestions, keywords.ideas) or the Google Ads API, depending on the selected mode and provider.

Data Transformation and Persistence

Mapping to EnrichedKeyword

Raw payloads from DataForSEO Labs are transformed into a stable internal shape through mapper functions in src/server/features/keywords/services/research/research-data.ts (lines 41-59). These functions convert the provider-specific JSON into EnrichedKeyword objects containing standardized fields: keyword text, search volume, CPC, competition level, keyword difficulty, search intent, and trend data.

Repository Persistence

After successful data fetching, each keyword metric is upserted into the database via KeywordResearchRepository (persistRows). This ensures historical tracking of keyword metrics across multiple research sessions.

Caching Layer

Results are cached using setCached from src/server/lib/r2-cache.ts. The cache layer uses the previously generated cache key to store serialized research results, enabling instant retrieval for identical subsequent requests and reducing API costs.

Implementation Examples

Calling the Server Function from React

The following TanStack React Start component demonstrates how to invoke the researchKeywords server function:

import { researchKeywords } from '@/serverFunctions/keywords';
import { useMutation } from '@tanstack/react-query';

function KeywordResearch({ projectId, keywords, location, language }) {
  const mutation = useMutation({
    mutationFn: (data) =>
      researchKeywords.mutateAsync({
        projectId,
        keywords,
        locationCode: location,
        languageCode: language,
        resultLimit: 50,
        mode: 'auto',
        clickstream: true,
      }),
  });

  const start = () => mutation.mutate();

  return (
    <div>
      <button onClick={start} disabled={mutation.isLoading}>
        Research Keywords
      </button>
      {mutation.isSuccess && (
        <ul>
          {mutation.data.rows.map((row) => (
            <li key={row.keyword}>
              {row.keyword} – {row.searchVolume ?? 'N/A'} vol, {row.intent}
            </li>
          ))}
        </ul>
      )}
    </div>
  );
}

Manual Mode with Suggestions

To fetch only search suggestions for a specific seed keyword:

await researchKeywords.mutateAsync({
  projectId,
  keywords: ['photoshop tutorial'],
  locationCode: 2840,            // United States
  languageCode: 'en',
  resultLimit: 30,
  mode: 'suggestions',           // forces the "suggestions" source
  clickstream: false,
});

Debugging Cache Keys

For troubleshooting cache hits and misses, inspect the generated cache key:

import { buildCacheKey } from '@/server/lib/r2-cache';

const key = await buildCacheKey('kw:research', {
  cacheVersion: 3,
  organizationId: 'org_123',
  projectId: 'proj_456',
  keywords: ['seo audit'],
  locationCode: 2840,
  languageCode: 'en',
  resultLimit: 20,
  mode: 'auto',
  depth: 3,
  clickstream: true,
});
console.log('Cache key →', key);

Summary

• Entry Point: The researchKeywords server function in src/serverFunctions/keywords.ts validates requests and delegates to KeywordResearchService.
• Normalization: Keywords are trimmed and lower-cased via normalizeKeyword before processing to ensure cache consistency.
• Multi-Source Strategy: Auto mode iterates through AUTO_KEYWORD_SOURCES (related → suggestions → ideas) until collecting 5 non-seed keywords.
• Provider Logic: getKeywordDataProvider routes requests to DataForSEO Labs or Google Ads based on market availability.
• Data Integrity: Raw API responses are mapped to EnrichedKeyword objects and persisted via KeywordResearchRepository while results are cached in R2.

Frequently Asked Questions

How does OpenSEO handle duplicate keywords in research requests?

OpenSEO deduplicates keywords using the normalizeKeyword helper in src/server/features/keywords/services/research/helpers.ts. This function trims whitespace and converts all keywords to lowercase before building cache keys and querying data providers, ensuring that "SEO Tools" and "seo tools" are treated as identical terms and cached under the same key.

What is the difference between auto mode and manual mode in OpenSEO keyword research?

Auto mode automatically iterates through multiple data sources (related, suggestions, ideas) in priority order defined in src/server/features/keywords/services/research/selection.ts until it collects at least five non-seed keywords. Manual mode bypasses this logic and queries only the specific source specified (e.g., 'suggestions'), giving developers precise control over which API endpoints are consumed.

How does OpenSEO optimize API costs when performing keyword research?

The system implements a coverage threshold of five non-seed keywords in auto mode, stopping additional API calls once sufficient data is collected. Additionally, deterministic cache keys generated by buildResearchCacheKey ensure identical requests return cached results instantly from src/server/lib/r2-cache.ts, eliminating redundant calls to DataForSEO or Google Ads.

Which data providers does OpenSEO use for keyword metrics?

According to the source code in src/server/features/keywords/services/research/research.ts, OpenSEO primarily uses DataForSEO Labs for organic keyword data (related keywords, suggestions, ideas) and the Google Ads API for advertising-specific metrics. The getKeywordDataProvider function selects the appropriate provider based on the target location, with fallback logic for Google-Ads-only markets.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}