How OmniRoute's Fusion Strategy Combines Model Panels and Judge Synthesis

OmniRoute's fusion routing strategy fans out requests to a panel of expert models, collects their independent answers, and uses a designated judge model to synthesize a single authoritative response—unlike other strategies that select only one target model.

This approach lets you leverage multiple LLM strengths simultaneously rather than betting on a single model. The fusion combo, implemented in open-sse/services/fusion.ts, is orchestrated through the combo dispatcher in open-sse/services/combo/dispatchPrelude.ts.

How Fusion Panel Construction Works

The fusion process begins with panel construction. The combo's models list is resolved into concrete ResolvedComboTarget objects during the prelude phase in dispatchPrelude.ts (lines 14-20).

Panel resolution follows these rules:

• Each model string in the combo config is validated and resolved to its provider and identifier
• The resulting array becomes your expert panel
• Maximum panel size is enforced at 40 models (configurable via FUSION_DEFAULTS.maxPanel)
• Oversized panels trigger an immediate 400 error before any network calls occur

Judge Model Selection and Role

The judge model is selected through a two-tier fallback system in handleFusionChat (lines 31-33 of fusion.ts):

1. Explicit configuration — If config.judgeModel is specified, that model always judges
2. Default fallback — The first panel model becomes the judge

This design ensures every fusion request has a deterministic judge even without explicit configuration.

Special Handling for Tool-Bearing Requests

When a request contains tools and tool_choice is not "none", panel synthesis would strip tool capability from individual panel calls. Fusion handles this by bypassing the panel entirely and routing directly to the judge with the original body intact (lines 123-130).

This preserves function-calling capabilities while still allowing the configured judge to process the request.

Parallel Panel Execution with Quorum-Grace Collection

The fan-out implementation executes panel models in parallel with sophisticated collection semantics:

Original body → strip tool fields → force stream: false → dispatch to all panel models

Each panel call uses dispatchFusionModel → handleSingleModel with a hard timeout (panelHardTimeoutMs). Results feed into collectPanel, which implements quorum-grace semantics (lines 174-186):

• Once minPanel successful answers arrive, a grace timer (stragglerGraceMs) starts
• Collection stops when the grace timer expires, even if additional responses are in-flight
• This balances answer quality against latency

Response Extraction and Normalization

Panel responses vary by provider format. The extractPanelText function (lines 48-81) normalizes:

• OpenAI chat completions
• Claude messages
• Gemini content
• OpenAI Responses API format

Each returns raw answer text for judge synthesis.

Graceful Degradation Paths

Fusion implements three degradation strategies based on panel results:

Result Count	Behavior
Zero answers	503 error with per-model failure descriptions
One answer	Direct return (no explicit judge) or judge polish (explicit judge configured)
Oversized panel ( >40 )	400 rejection before fan-out

Judge Synthesis and Prompt Engineering

The judge synthesis stage transforms panel outputs into a final answer through careful prompt construction in buildJudgePrompt:

1. Anonymization — Panel texts labeled as "Source N" to prevent judge bias toward specific models
2. System instruction — Directs the judge to analyze:
   • Consensus across sources
   • Contradictions requiring resolution
   • Partial coverage gaps
   • Unique insights from individual sources
   • Potential blind spots
3. Output constraint — Final answer must not mention sources by name

The judge request uses appendUserTurn to add the synthesized prompt as a new user message, then dispatches via handleSingleModel with original stream flag and tools preserved (lines 44-51).

Tunable Fusion Parameters

Fusion behavior is controlled through combo configuration, documented in docs/routing/AUTO-COMBO.md (lines 262-311):

Parameter	Default	Purpose
minPanel	2	Minimum successful responses to trigger grace period
stragglerGraceMs	8000	Grace timer for late panel responses
panelHardTimeoutMs	90000	Per-model timeout ceiling
maxPanel	40	Maximum panel size before rejection

Practical Fusion Combo Example

{
  "model": "fusion-panel",
  "messages": [
    { "role": "user", "content": "Explain the differences between HTTP/2 and HTTP/3." }
  ],
  "stream": false,
  "config": {
    "judgeModel": "openai/gpt-4o-mini",
    "fusionTuning": {
      "minPanel": 2,
      "stragglerGraceMs": 8000,
      "panelHardTimeoutMs": 90000
    }
  },
  "models": [
    "openai/gpt-4o-mini",
    "claude/claude-3-5-sonnet-20241022",
    "gemini/gemini-2.5-flash"
  ]
}

The integration test in tests/integration/combo-matrix/fusion.test.ts (lines 25-44) demonstrates this flow, asserting that three panel calls plus one judge call complete successfully.

Key Implementation Files

• open-sse/services/fusion.ts — Core fusion logic: panel fan-out, quorum-grace collection, judge prompt building, error handling
• open-sse/services/combo/dispatchPrelude.ts — Panel resolution and judge selection prelude
• docs/routing/AUTO-COMBO.md — Parameter documentation and configuration examples
• tests/integration/combo-matrix/fusion.test.ts — Integration tests validating end-to-end behavior
• open-sse/services/combo/types.ts — Type definitions for ResolvedComboTarget and related interfaces

Summary

• Fusion is OmniRoute's only strategy that does not select a single model, instead aggregating multiple expert opinions
• Panel construction happens in dispatchPrelude.ts with hard limits on panel size
• Judge selection prioritizes explicit configuration, falling back to the first panel model
• Tool-bearing requests bypass panel synthesis to preserve function-calling capabilities
• Quorum-grace collection balances answer completeness against latency
• The judge prompt anonymizes sources and instructs synthesis without attribution
• All parameters are tunable through combo configuration with sensible defaults

Frequently Asked Questions

What happens if all panel models fail in a fusion request?

OmniRoute returns a 503 error with detailed per-model failure descriptions. This graceful degradation ensures clients receive actionable diagnostic information rather than a generic failure.

Can I use different models for the panel and judge in fusion?

Yes. Set config.judgeModel to any valid model identifier. The panel models listed in models serve as experts, while the designated judge handles synthesis. If you omit judgeModel, the first panel model serves double duty.

How does fusion handle streaming responses?

Fusion disables streaming for panel calls (forces stream: false) to enable proper response collection and text extraction. The final judge response respects the original client's stream preference, so you can still receive streamed output from the synthesized answer.

Why does fusion have a maximum panel size of 40?

The maxPanel default of 40 prevents resource exhaustion and excessive latency. Each panel member requires a network call and judge context window space. Panels exceeding this limit receive an immediate 400 rejection before any network activity, protecting both the server and client from runaway requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}