Fusion Routing Strategy in OmniRoute: Parallel Model Execution with Judge Synthesis

The fusion routing strategy executes requests against a panel of AI models in parallel, then synthesizes their responses through a judge model to produce a single authoritative answer.

OmniRoute's fusion routing strategy is a sophisticated combo mode designed to improve response quality by combining outputs from multiple independent models. Implemented in the open-sse/services/fusion.ts module, this strategy leverages parallel execution, quorum-based collection, and intelligent synthesis to balance accuracy against latency.

How Fusion Works: The 7-Step Pipeline

The fusion strategy operates through a carefully orchestrated sequence that handles everything from panel dispatch to failure recovery.

1. Panel Fan-Out to Multiple Models

When a request arrives for a fusion combo, the strategy immediately strips tool information (stream: false) and dispatches the request body to every model in the combo's models array. This parallel dispatch is handled by dispatchFusionModel at lines 45-48 in fusion.ts.

The fan-out creates redundant execution paths, ensuring that slowdowns or failures in individual models don't block the entire request.

2. Quorum-Grace Collection

Rather than waiting for all panel members, fusion uses quorum-grace collection to minimize latency. The collectPanel function (lines 82-118) implements this behavior:

• As soon as minPanel members succeed, a grace timer begins
• The stragglerGraceMs window (default 8,000ms) allows slower models to finish
• Collection stops when the timer expires, preventing a single slow model from dominating latency

This approach trades completeness for speed while maintaining sufficient diversity for the judge model.

3. Failure Handling and Error Classification

Panel members that timeout, throw exceptions, or return non-OK HTTP statuses are recorded with specific failure reasons. The failure handling loop (lines 91-138) captures:

• timeout — member exceeded time limits
• status_429 — rate limiting from provider
• Other HTTP status codes as status_{code}

If no panel member succeeds, the entire combo returns 503 Service Unavailable with a detailed error message explaining which members failed and why.

4. Judge Synthesis with Anonymous Sources

Once at least one answer is collected, the judge model receives a synthesized request. Two key functions handle this:

• buildJudgePrompt (lines 22-40) — crafts a prompt that anonymizes sources as "Source 1", "Source 2", etc.
• appendUserTurn (lines 4-15) — injects this prompt as a new user message

The judge is instructed to produce one authoritative answer that may improve upon the panel consensus, avoiding simple voting or averaging.

5. Tool-Bearing Request Bypass

Fusion intelligently handles requests containing tools. The isToolBearingRequest check (lines 50-54) detects when a request has tools and tool_choice is not "none". In this case:

• The entire fusion panel is skipped
• The request routes directly to the judge (or first panel member)
• Original tools remain intact

This prevents stripping tools from models that require them for function calling.

6. Panel Size Protection

To prevent out-of-memory crashes, a hard limit (maxPanel, default 40) rejects oversized panels before any fan-out occurs. The size guard at lines 90-99 returns 400 Bad Request with a clear message:

"Fusion panel too large (50 models, max 40) — reduce the combo's target count …"

7. Per-Target Admission Control

An optional perTargetAdmission hook (lines 21-46) allows fine-grained capacity management. This hook can drop panel members that exceed lane capacity, ensuring only surviving members participate in quorum calculation and judge selection.

Configuration and Defaults

Fusion behavior is controlled through combo.config.fusionTuning, with these default tuning values:

export const FUSION_DEFAULTS = {
  minPanel: 2,           // Minimum successful responses to trigger judge
  stragglerGraceMs: 8_000,  // Wait for stragglers after quorum
  panelHardTimeoutMs: 90_000,  // Absolute ceiling for panel collection
  maxPanel: 40,          // Hard limit on panel size
} as const;

The strategy is registered in the global routing strategy enumeration at src/shared/constants/routingStrategies.ts:

{
  value: "fusion",
  labelKey: "fusion",
  combosDescKey: "fusionDesc",
}

Practical Implementation Examples

Configuring a Fusion Combo

{
  "name": "my-fusion-combo",
  "strategy": "fusion",
  "models": [
    "openai/gpt-4o",
    "anthropic/claude-3-5-sonnet",
    "google/gemini-1.5-pro"
  ],
  "config": {
    "judgeModel": "openai/gpt-4o",
    "fusionTuning": {
      "minPanel": 2,
      "stragglerGraceMs": 6000,
      "maxPanel": 30
    }
  }
}

Direct Handler Integration

import { handleFusionChat } from "@/open-sse/services/fusion.ts";
import { handleSingleModel } from "@/open-sse/services/combo/types.ts";

async function myFusionEndpoint(req: Request) {
  const body = await req.json();
  const response = await handleFusionChat({
    body,
    models: [
      "openai/gpt-4o",
      "anthropic/claude-3-5-sonnet",
      "google/gemini-1.5-pro",
    ],
    handleSingleModel,
    log: console,
    comboName: "my-fusion-combo",
    judgeModel: "openai/gpt-4o",
    tuning: { minPanel: 2 },
  });
  return response;
}

Tool-Aware Routing Detection

// Automatically detected by fusion internals — no manual check needed
// When present, tools bypass the panel and route with original payload

Key Source Files

File	Purpose
open-sse/services/fusion.ts	Core implementation: panel fan-out, quorum-grace, judge synthesis, error handling
src/shared/constants/routingStrategies.ts	Strategy registry entry for "fusion"
open-sse/services/combo/fusionPanel.ts	Helper for fusion combo definition generation
open-sse/utils/error.ts	errorResponse and sanitizeErrorMessage utilities
open-sse/translator/helpers/geminiHelper.ts	extractTextContent for panel response processing

Summary

• Fusion routing strategy parallelizes requests across a panel of models, then synthesizes outputs through a judge model for improved response quality
• Quorum-grace collection (minPanel + stragglerGraceMs) balances completeness against latency
• Tool-bearing requests bypass the panel entirely to preserve function-calling capabilities
• Hard limits (maxPanel, panelHardTimeoutMs) protect against resource exhaustion
• Fine-grained tuning via fusionTuning configuration per combo
• Core implementation lives in open-sse/services/fusion.ts with handleFusionChat as the primary entry point

Frequently Asked Questions

What happens if all panel members fail in a fusion request?

If no panel member succeeds, fusion returns 503 Service Unavailable with a detailed error message listing each member's failure reason (timeout, rate limit, HTTP error status, etc.). This ensures callers receive actionable diagnostic information rather than an opaque failure.

Can I use fusion with models that require tool calling?

Yes, but with automatic adaptation. When isToolBearingRequest detects tools with tool_choice !== "none", fusion skips the panel entirely and routes directly to the judge or first panel member with tools intact. This prevents the tool-stripping that occurs during normal panel fan-out.

How do I reduce latency when some models are consistently slow?

Lower stragglerGraceMs in your combo's fusionTuning configuration. The default 8,000ms grace window can be reduced to 2,000-4,000ms for latency-sensitive applications. Alternatively, increase minPanel if your use case can tolerate fewer diverse perspectives before triggering judge synthesis.

Is there a limit to how many models I can include in a fusion panel?

Yes, the default maxPanel is 40 models, enforced before any fan-out begins. This prevents memory exhaustion from excessive parallel requests. You can raise this limit via fusionTuning.maxPanel, but monitor resource usage carefully as each panel member consumes concurrent request capacity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}