How OpenSEO Keyword Research Endpoints Aggregate Suggestions and Ideas

OpenSEO's keyword research endpoints aggregate suggestions by fetching data from DataForSEO, saved project keywords, and Google Search Console, then normalizing, deduplicating, scoring, and caching the results before returning a unified list.

OpenSEO (available at every-app/open-seo) is an open-source SEO platform that consolidates keyword intelligence from multiple sources into actionable recommendations. Understanding how its keyword research endpoints aggregate suggestions reveals a sophisticated pipeline that balances external API data with proprietary user data. The system normalizes disparate formats, removes duplicates, and applies a custom scoring algorithm to surface high-value keywords.

The Keyword Aggregation Pipeline

API Entry Point and Input Validation

The aggregation flow begins at the GET /api/keyword-research route defined in [src/routes/api/keyword-research.ts](https://github.com/every-app/open-seo/blob/main/src/routes/api/keyword-research.ts). This endpoint accepts seed terms, competitor domains, or Google Search Console query IDs as input parameters.

Input validation occurs through Zod schemas located in [src/lib/validation.ts](https://github.com/every-app/open-seo/blob/main/src/lib/validation.ts). The validation layer ensures that seed keywords are properly formatted and that project identifiers match existing records before any external API calls are initiated.

Service Orchestration Across Three Data Sources

Once validated, the request delegates to KeywordService in [src/services/keyword-service.ts](https://github.com/every-app/open-seo/blob/main/src/services/keyword-service.ts). This service orchestrates three parallel fetch operations to gather comprehensive keyword intelligence:

1. DataForSEO "Keyword Ideas" – The DataForSEOClient.getKeywordIdeas(seeds) method in [src/lib/dataforseo.ts](https://github.com/every-app/open-seo/blob/main/src/lib/dataforseo.ts) retrieves search volume, keyword difficulty, CPC, and raw relevance scores from the DataForSEO API.

2. Saved Project Keywords – ProjectRepository.getSavedKeywords(projectId) in [src/repositories/project-repo.ts](https://github.com/every-app/open-seo/blob/main/src/repositories/project-repo.ts) queries previously saved keywords associated with the user's project, including historical metric fields.

3. Google Search Console Queries – GSCClient.getQueries(projectId) in [src/lib/gsc.ts](https://github.com/every-app/open-seo/blob/main/src/lib/gsc.ts) fetches queries where the site already ranks, complete with click-through statistics and impression data.

These three sources execute concurrently via Promise.all(), minimizing latency before the aggregation phase begins.

Data Normalization and Type Unification

Raw results from each source vary in schema and metric availability. The pipeline normalizes these into a common KeywordSuggestion type defined in [src/types/keyword.ts](https://github.com/every-app/open-seo/blob/main/src/types/keyword.ts).

Each fetcher result passes through dedicated normalization functions that coerce fields into a unified shape. During this phase, the system calculates a "temperature" metric derived from keyword difficulty, CPC, and recent traffic trends, ensuring comparability across disparate data origins.

Deduplication and Scoring Algorithm

The core aggregation logic resides in KeywordAggregator.aggregate() within [src/lib/keyword-aggregator.ts](https://github.com/every-app/open-seo/blob/main/src/lib/keyword-aggregator.ts). This class merges the three normalized lists and eliminates duplicates based on the keyword string.

For scoring, OpenSEO applies the following algorithm to each unique suggestion:

const score = suggestion.volume * (1 - suggestion.difficulty) + suggestion.cpc * 0.1;

The aggregator maintains a Map<string, KeywordSuggestion> to track the highest-scoring variant of each keyword. When duplicate keywords appear across sources, the implementation retains only the version with the superior score. Finally, the deduplicated array sorts in descending order by this composite score, prioritizing high-volume, low-difficulty opportunities.

Caching Layer

To prevent redundant external API calls, results are cached in D1/Redis via [src/lib/cache.ts](https://github.com/every-app/open-seo/blob/main/src/lib/cache.ts). The endpoint stores aggregated responses for 5 minutes, significantly reducing DataForSEO API usage while maintaining reasonably fresh data. Client applications should align their stale-time configurations with this server-side TTL.

Implementation Example

When calling the endpoint from a frontend application, implement a query hook that respects the 5-minute cache window:

import { useQuery } from '@tanstack/react-query';

function useKeywordIdeas(seeds: string[]) {
  return useQuery(
    ['keyword-ideas', seeds],
    async () => {
      const resp = await fetch(
        `/api/keyword-research?seeds=${encodeURIComponent(seeds.join(','))}`
      );
      if (!resp.ok) throw new Error('Failed to fetch keyword ideas');
      return resp.json();               // { suggestions: [...], sourceBreakdown: {...} }
    },
    { staleTime: 300_000 } // 5 minutes, matches server-side cache
  );
}

On the server, the KeywordService orchestrates the parallel fetch and aggregation:

// src/services/keyword-service.ts
export async function getAggregatedKeywords(
  opts: KeywordRequestOpts,
): Promise<KeywordSuggestion[]> {
  const [dataForSEO, saved, gsc] = await Promise.all([
    dataForSEOClient.getKeywordIdeas(opts.seeds),
    projectRepo.getSavedKeywords(opts.projectId),
    gscClient.getQueries(opts.projectId),
  ]);

  const normalized = [
    ...dataForSEO.map(normalizeDataForSEO),
    ...saved.map(normalizeSaved),
    ...gsc.map(normalizeGSC),
  ];

  return KeywordAggregator.aggregate(normalized);
}

The KeywordAggregator class handles the deduplication and ranking logic:

// src/lib/keyword-aggregator.ts
export class KeywordAggregator {
  static aggregate(
    suggestions: KeywordSuggestion[],
  ): KeywordSuggestion[] {
    const map = new Map<string, KeywordSuggestion>();

    for (const sug of suggestions) {
      const existing = map.get(sug.keyword);
      if (!existing || sug.score > existing.score) {
        map.set(sug.keyword, sug);
      }
    }

    return Array.from(map.values())
      .sort((a, b) => b.score - a.score);
  }
}

Summary

• Multi-source ingestion: OpenSEO aggregates keywords from DataForSEO APIs, project databases, and Google Search Console simultaneously.
• Strict normalization: All inputs convert to the unified KeywordSuggestion type in src/types/keyword.ts to ensure field consistency.
• Intelligent deduplication: The KeywordAggregator class in src/lib/keyword-aggregator.ts maintains only the highest-scoring variant when keywords appear across multiple sources.
• Composite scoring: The ranking algorithm weighs search volume, keyword difficulty, and CPC to surface actionable opportunities.
• Performance optimization: A 5-minute D1/Redis cache in src/lib/cache.ts minimizes external API costs while maintaining data freshness.

Frequently Asked Questions

How does OpenSEO handle duplicate keywords across different data sources?

When the same keyword appears in DataForSEO results, saved project data, and Google Search Console, the KeywordAggregator.aggregate() method in src/lib/keyword-aggregator.ts stores entries in a Map keyed by the keyword string. It compares the calculated scores using the formula volume * (1 - difficulty) + cpc * 0.1 and retains only the highest-scoring instance, ensuring the final list contains unique recommendations with the best available metrics.

What data sources does the keyword research endpoint use?

According to the source code in src/services/keyword-service.ts, the endpoint queries three distinct sources: the DataForSEO "Keyword Ideas" API via src/lib/dataforseo.ts, the project's saved keyword repository via src/repositories/project-repo.ts, and Google Search Console query data via src/lib/gsc.ts. These sources provide external market data, historical user data, and actual performance metrics respectively.

How is the keyword relevance score calculated?

OpenSEO calculates a composite score using the formula suggestion.volume * (1 - suggestion.difficulty) + suggestion.cpc * 0.1 as implemented in src/lib/keyword-aggregator.ts. This weights search volume heavily while penalizing high difficulty scores, with cost-per-click data providing a minor boost to commercial intent keywords. The system then sorts the final array descending by this score.

How long are keyword research results cached?

The endpoint caches aggregated results for 5 minutes using the caching layer defined in src/lib/cache.ts, which supports both D1 and Redis backends. This TTL balances data freshness with API rate limit conservation, and the server returns a sourceBreakdown map in the response to indicate which sources contributed to the cached result.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}