How Wigolo Implements Multi-Engine Search with Rank Fusion and ML Reranking

Wigolo combines parallel search engine dispatch, reciprocal-rank fusion, and optional transformer-based reranking to deliver highly relevant results from multiple sources.

This deep dive into the KnockOutEZ/wigolo repository explains how the search orchestrator merges results from engines like Bing and DuckDuckGo using a mathematically grounded fusion algorithm, then optionally applies machine learning to refine the final ranking.

Parallel Engine Dispatch Architecture

The search flow begins in src/tools/search.ts, where the handleSearch function validates incoming requests and forwards them to the core provider.

// src/tools/search.ts
export async function handleSearch(
  input: SearchInput,
  engines: SearchEngine[],
  router: SmartRouter,
  …,
): Promise<StageResult<SearchOutput>> {
  const provider = await getSearchProvider();          // core provider or legacy SearXNG
  return provider.search(input, { engines, router, … });
}

The provider ultimately delegates to runV1Search in src/search/core/orchestrator.ts, which executes all enabled engines in parallel. This stage respects soft deadlines defined by constants like ENGINE_POOL_SOFT_DEADLINE_MS and ENGINE_POOL_CHRONIC_SOFT_DEADLINE_MS, ensuring slow engines do not block the entire result set.

Reciprocal-Rank Fusion (RRF) Implementation

Once raw results return from each engine, the orchestrator deduplicates them per-engine before applying weighted Reciprocal-Rank Fusion (RRF) inside the scoreOutcomes function.

// src/search/core/orchestrator.ts – scoreOutcomes()
const base = weight / (RRF_K + rank);
fused.set(key, (fused.get(key) ?? 0) + base * recMul);

Key implementation details include:

• RRF_K constant: Fixed at 60 (const RRF_K = 60;), this denominator prevents high rankings from dominating the final score.
• Engine weights: Resolved dynamically by resolveEngineWeight in src/search/core/engine-quality.ts, allowing high-quality sources to contribute more heavily to the fused score.
• Preliminary scoring: The fused output becomes the initial relevance_score for every URL.

Post-Fusion Boosts and Guards

Before finalizing the RRF output, the orchestrator applies several domain-aware heuristics:

• Authority boost: applyAuthorityBoost in src/search/reranker/authority-boost.ts elevates results from trusted domains.
• Recency multiplier: Fresh content receives a temporal boost via recencyMultiplier.
• Brand-collision guard: applyBrandCollisionGuard prevents over-representation of single domains.
• Rare-term handling: Lexical alignment bonuses improve visibility for uncommon query terms.

ML-Based Reranking Pipeline

When configuration enables reranking (reranker !== 'none'), the orchestrator calls rerankResults in src/search/rerank.ts to reorder the fused list using a transformer model.

// src/search/rerank.ts (excerpt)
const provider = await getRerankProvider();   // loads the ML model
const reranked = await provider.rerank(query, results);
// reranked results replace the order produced by RRF

The TransformersRerankProvider located in src/search/reranker/transformers-rerank-provider.ts loads a lightweight bi-encoder (ONNX or HuggingFace) that scores query-result pairs for semantic relevance. The final relevance_score is then normalized to a 0-1 scale unless the engine pool degrades below the RANK_DEGRADED_CONFIDENCE_FLOOR of 0.05.

Each result includes an evidence_score object explaining the contribution of every component—RRF base, domain quality, lexical alignment, and recency—providing transparency into the ranking decision.

Practical Implementation Examples

Basic Multi-Engine Search via SDK

The following TypeScript example demonstrates both standard RRF fusion and ML-enhanced reranking:

import { createClient } from '@wigolo/sdk';

const client = createClient({ apiKey: process.env.WIGOLO_API_KEY });

async function demo() {
  // 1️⃣ Simple multi-engine search – uses RRF fusion only
  const res1 = await client.search({ query: 'typescript async await' });
  console.log('RRF-fused results:', res1.results.map(r => r.title));

  // 2️⃣ Enable the ML reranker (requires the model to be warmed-up)
  const res2 = await client.search({
    query: 'typescript async await',
    reranker: 'onnx',               // or 'transformers' depending on config
  });
  console.log('ML-reranked results:', res2.results.map(r => r.title));
}
demo();

Custom Search with Explicit Engine Selection

For advanced use cases, bypass the high-level client and invoke handleSearch directly with a curated engine list:

import { SearchEngine } from '@wigolo/sdk/types';
import { handleSearch } from './src/tools/search.js';
import { SmartRouter } from './src/fetch/router.js';

const router = new SmartRouter();                     // manages proxies & TLS tiers
const engines: SearchEngine[] = ['bing', 'duckduckgo']; // pick any subset

const out = await handleSearch(
  { query: 'open source licenses' },
  engines,
  router,
);
console.log(out.results.map(r => `${r.title} – ${r.url}`));

Summary

• Entry point: handleSearch in src/tools/search.ts routes requests to the core orchestrator.
• Parallel execution: runV1Search dispatches all enabled engines simultaneously with deadline-aware pooling.
• Rank fusion: The RRF algorithm (scoreOutcomes) merges results using RRF_K = 60 and engine-specific weights from src/search/core/engine-quality.ts.
• Boosting layer: Authority, recency, and brand-collision guards refine the preliminary RRF scores.
• Optional ML reranking: rerankResults leverages TransformersRerankProvider to reorder results by semantic relevance when enabled.
• Resilience: Fallback logic retries general verticals when specific engine pools fail, and confidence floors prevent degraded results from skewing rankings.

Frequently Asked Questions

What is the RRF_K constant and why is it set to 60?

RRF_K is a smoothing constant in the reciprocal-rank fusion formula that prevents top-ranked items from overwhelming the final score. Wigolo hardcodes this value at 60 in src/search/core/orchestrator.ts based on established information retrieval research, ensuring balanced contribution across engines regardless of their individual result depths.

How does Wigolo handle slow or failing search engines?

The orchestrator implements soft deadlines via ENGINE_POOL_SOFT_DEADLINE_MS and chronic timeouts, allowing fast engines to return results while isolating sluggish sources. If a vertical-specific engine pool collapses below the RANK_DEGRADED_CONFIDENCE_FLOOR of 0.05, the system falls back to general search verticals to maintain result coverage.

Can I disable the ML reranker and rely solely on RRF fusion?

Yes, setting reranker: 'none' in the search configuration skips the transformer-based reordering entirely. In this mode, Wigolo returns results ranked purely by the RRF fusion score plus authority and recency boosts, which is optimal for low-latency applications where model inference overhead is unacceptable.

Where are engine quality weights defined?

Per-engine weights are resolved by resolveEngineWeight in src/search/core/engine-quality.ts, allowing the system to mathematically trust high-quality sources (like authoritative indexes) more than smaller engines during the fusion calculation. These weights directly multiply the reciprocal-rank score in the RRF formula.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}