OmniRoute Fusion Routing Strategy: How Judge Models Synthesize Multi-Panel Responses

OmniRoute's fusion routing strategy fans out requests to a panel of expert models in parallel and uses a dedicated judge model to synthesize a single, coherent answer from their collective outputs.

OmniRoute is an open-source LLM routing layer that supports multiple combo routing strategies. Among these, the OmniRoute fusion routing strategy is unique because it does not select a single winner from its model pool; instead, it orchestrates a multi-stage pipeline where a panel of "expert" models generates independent responses, and a judge model aggregates them into a final, unified result. This approach maximizes coverage across different model strengths while mitigating individual model weaknesses.

How the Fusion Strategy Works

Parallel Panel Execution

When the combo dispatcher receives a request with strategy: "fusion", it constructs a fusionPanel array containing the selected expert models. Unlike other routing modes, the dispatcher strips tool definitions from these panel requests to ensure the judge receives plain prose for synthesis. The system then fires non-streaming requests to every model in the panel simultaneously, as implemented in open-sse/services/fusion.ts.

Judge Model Selection

The judge responsible for final synthesis can be explicitly configured via combo.config.judgeModel. If omitted, the system defaults to using the first model in the fusionPanel as the implicit judge. This selection logic resides in the tuning parameter initialization section of the fusion service.

Synthesis and Tuning Parameters

The fusion engine collects panel responses and waits for a configurable quorum before engaging the judge. Three key tuning parameters control this behavior:

• minPanel: The minimum number of successful panel answers required before the grace timer starts (default: 2).
• stragglerGraceMs: The duration to wait for lagging panel models after the quorum is reached (default: 8000 ms).
• panelHardTimeoutMs: The absolute upper bound on total fan-out time to prevent hung models from blocking the request (default: 90,000 ms).

If the quorum is met, the judge receives the anonymized responses; otherwise, the system may fall back to the first successful panel answer.

The Role of the Judge Model

Aggregation and Synthesis

The judge acts as an intelligent aggregator. It receives the original conversation context (system prompt plus user turn) appended with a specialized judge prompt that enumerates anonymized panel outputs labeled as "Source 1", "Source 2", etc. This design allows the judge to combine disparate opinions, reconcile contradictions, and produce a coherent narrative that leverages the collective knowledge of the panel.

Failure Recovery and Override

Starting with version 3.8.50, the judge model can override panel failures. If every panel model returns incorrect, incomplete, or hallucinated responses, the judge draws upon its own parametric knowledge to generate a valid answer rather than synthesizing garbage. This safety mechanism ensures robustness even when the entire panel underperforms.

Unbiased Anonymization

To prevent the judge from favoring specific models based on brand recognition or response style, OmniRoute anonymizes panel sources. The judge prompt refers only to "Source N" identifiers, ensuring the synthesis process remains objective and quality-based rather than vendor-biased.

Implementation Architecture

The fusion logic lives in open-sse/services/fusion.ts and is invoked from the combo dispatcher in open-sse/services/combo.ts via the handleFusionChat function. The flow is orchestrated through open-sse/services/combo/dispatchPrelude.ts, which extracts fusion-specific configuration and delegates to the fusion handler. The final judge request is executed through handleSingleModel(judgeBody, effectiveJudge), returning the synthesized response to the client.

Configuring a Fusion Combo

Define a fusion combo by posting to the /v1/combo endpoint:

{
  "name": "fusion-panel",
  "strategy": "fusion",
  "models": ["gpt-4o-mini", "claude-3-sonnet-20240229", "gemini-1.5-flash"],
  "config": {
    "judgeModel": "gpt-4o-mini",
    "fusionTuning": {
      "minPanel": 2,
      "stragglerGraceMs": 8000,
      "panelHardTimeoutMs": 90000
    }
  }
}

Send requests to this combo via the standard completions endpoint:

{
  "model": "fusion-panel",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Explain the difference between HTTP/2 and HTTP/3." }
  ],
  "stream": false
}

The panel models execute in parallel, and the judge receives a prompt structured like:

You are the JUDGE in a model-fusion panel. 3 expert models independently answered the user's most recent request. Their responses are below, anonymized by source.

Source 1:
...
Source 2:
...
Source 3:
...

The judge synthesizes a single answer that is returned to the client.

Summary

• OmniRoute's fusion routing strategy is the only combo mode that uses multiple models simultaneously rather than selecting one winner.
• The panel fan-out executes requests in parallel to several expert models, with tools stripped to ensure clean prose inputs.
• A judge model synthesizes the panel outputs, configured explicitly via judgeModel or defaulting to the first panel member.
• Tuning parameters (minPanel, stragglerGraceMs, panelHardTimeoutMs) control quorum requirements and timeout behavior.
• The judge can override panel failures and operates on anonymized sources to eliminate bias.
• Core implementation resides in open-sse/services/fusion.ts and integrates with the combo system via handleFusionChat.

Frequently Asked Questions

What makes OmniRoute's fusion strategy different from other combo routing modes?

Unlike strategies that select a single model based on cost or latency, the fusion strategy fans out to a panel of models and uses a judge to synthesize their outputs. This provides ensemble coverage rather than single-model selection, allowing the system to combine strengths from heterogeneous models.

How does the judge model handle incomplete or incorrect panel responses?

Since version 3.8.50, the judge can override the panel entirely. If all panel responses are erroneous or incomplete, the judge relies on its own internal knowledge to generate a correct answer rather than propagating the errors, effectively acting as a safety net for the fusion pipeline.

Can I specify any model as the judge, or does it have to be one of the panel models?

You can specify any available model via combo.config.judgeModel. If you omit this configuration, the system automatically uses the first model listed in the fusionPanel as the implicit judge, though using a distinct, potentially stronger model as judge is recommended for optimal synthesis quality.

What happens if the panel doesn't reach the minimum quorum before timeouts?

If the number of successful panel responses falls below minPanel before panelHardTimeoutMs expires, the system may fall back to returning the first successful panel answer rather than waiting indefinitely or failing the request entirely, ensuring reasonable latency even under degraded conditions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}