# How OmniRoute Fusion Routing Uses Model Panels and a Judge for LLM Consensus

> Discover how OmniRoute Fusion routing achieves LLM consensus using parallel model panels and a judge model for authoritative answers.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-13

---

**OmniRoute Fusion routing combines parallel model execution with a judge model to synthesize authoritative answers from multiple LLM perspectives.**

OmniRoute's **fusion combo strategy** implements a two-stage pipeline that distributes prompts across a panel of models, then delegates synthesis to a judge. This architecture balances **latency, reliability, and answer quality** by capping wait times for slow responders while still capturing diverse model outputs. According to the OmniRoute source code in [`open-sse/services/fusion.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/fusion.ts), the entire workflow is orchestrated through configurable timeouts, quorum rules, and graceful degradation paths.

## Stage 1: Panel Fan-Out for Parallel Execution

The fusion handler begins by broadcasting the incoming request to all models in the configured panel. This parallel dispatch is controlled by several tunable parameters that prevent resource exhaustion and bound latency.

### Non-Streaming, Tool-Stripped Panel Calls

Each panel invocation is forced into a **non-streaming mode** with tools removed:

- The `tools` and `tool_choice` parameters are stripped so panel models return plain prose responses
- Streaming is disabled to simplify response collection and timing
- Results are extracted via `extractPanelText`, which uses `extractTextContent` from [`open-sse/translator/helpers/geminiHelper.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/helpers/geminiHelper.ts)

This design ensures panel responses are **uniformly comparable** for the downstream judge.

### Quorum-Grace Collection with `collectPanel`

The panel gathering logic implements a **quorum-grace** mechanism to optimize latency:

| Parameter | Default | Purpose |
|-----------|---------|---------|
| `minPanel` | 2 | Minimum successful responses before considering early return |
| `stragglerGraceMs` | 5000ms | Wait time after quorum for slower models to finish |
| `panelHardTimeoutMs` | 30000ms | Absolute deadline for any single panel call |
| `maxPanel` | 40 | Maximum panel size to prevent OOM crashes |

Once `minPanel` answers arrive, the grace timer starts. When it expires—or the hard timeout fires—`collectPanel` returns whatever responses have accumulated. This **caps straggler penalty** without abandoning complete panels when all models respond quickly.

Oversized panels are rejected at the validation layer: `maxPanel` defaults to 40 and can be tuned lower for memory-constrained deployments.

## Stage 2: Judge Synthesis for Consensus Answers

After panel collection completes, the **judge model** produces the final response through structured prompt engineering and configurable model selection.

### Judge Model Selection Logic

The judge is determined by this priority order:

1. **Explicit `judgeModel`** in combo configuration — used if specified
2. **`panel[0]`** — the first panel model serves as default judge

This fallback ensures fusion combos work without extra configuration while allowing optimization for specific synthesis tasks.

### The Judge Prompt: Anonymized Consensus Analysis

The judge receives a synthesized request containing:

- The **original user turn** (preserved from the incoming request)
- A **judge prompt** built by `buildJudgePrompt`, which:
  - Labels each panel source as "Source N" (anonymized to prevent bias toward model brands)
  - Instructs analysis of **consensus, contradictions, partial coverage, unique insights, and blind spots**
  - Requires one **authoritative final answer**

The judge call **retains original streaming and tool configurations**, so the final response can stream to clients or invoke tools if the original request specified them.

```ts
// Example: configuring a fusion combo with custom judge
const myFusionCombo = {
  name: 'my-fusion',
  strategy: 'fusion',
  targets: ['openai/gpt-4', 'anthropic/claude-2', 'google/gemini-1.5'],
  judgeModel: 'openai/gpt-4-judge', // explicit judge overrides panel[0]
  fusionTuning: {
    minPanel: 2,
    stragglerGraceMs: 5000,
    panelHardTimeoutMs: 30000,
    maxPanel: 30,
  },
} as const;

```

## Graceful Degradation Paths

Fusion routing handles partial failures through three distinct paths defined in [`open-sse/services/fusion.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/fusion.ts):

- **Zero successful panel answers** → Returns HTTP 503 with per-model failure reasons for debugging
- **Exactly one success, no explicit judge** → Returns the single answer directly (fast path, no synthesis overhead)
- **One success with explicit judge configured** → Judge still synthesizes a polished response (quality over speed)

This tiered fallback ensures **availability without sacrificing quality** when explicitly requested.

## Core Implementation Files

| File | Role |
|------|------|
| [`open-sse/services/fusion.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/fusion.ts) | Main handler: `handleFusionChat`, `collectPanel`, `buildJudgePrompt`, `extractPanelText` |
| [`open-sse/services/combo/types.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/types.ts) | Type definitions: `ComboLogger`, `ResolvedComboTarget` |
| [`open-sse/utils/error.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/utils/error.ts) | Error formatting: `errorResponse`, `sanitizeErrorMessage` |
| [`open-sse/translator/helpers/geminiHelper.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/helpers/geminiHelper.ts) | Text extraction: `extractTextContent` |
| [`tests/unit/combo-fusion-strategy.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/combo-fusion-strategy.test.ts) | Validation of fan-out, synthesis, and degradation behaviors |

## Usage in the Combo Pipeline

```ts
import { handleFusionChat } from '@/open-sse/services/fusion';
import type { ComboLogger, ResolvedComboTarget } from '@/open-sse/services/combo/types';

async function routeFusion(
  body: Record<string, unknown>,
  panelModels: (ResolvedComboTarget | string)[],
  log: ComboLogger,
) {
  return handleFusionChat({
    body,
    models: panelModels,
    handleSingleModel: yourSingleModelDispatcher,
    log,
    comboName: 'my-fusion',
    judgeModel: 'openai/gpt-4-judge',
  });
}

```

The `handleSingleModel` parameter injects your existing single-model dispatcher, letting fusion reuse your standard routing infrastructure.

## Summary

- **Panel fan-out** executes requests in parallel with non-streaming, tool-stripped calls bounded by `panelHardTimeoutMs` and `maxPanel`
- **Quorum-grace collection** (`minPanel` + `stragglerGraceMs`) optimizes latency without sacrificing coverage
- **Judge synthesis** anonymizes sources as "Source N" and mandates consensus analysis before producing the final answer
- **Default judge fallback** to `panel[0]` simplifies configuration; explicit `judgeModel` overrides when specialized synthesis is needed
- **Three degradation paths** (503, direct return, or forced synthesis) balance availability and quality

## Frequently Asked Questions

### How does OmniRoute Fusion prevent slow models from blocking responses?

Fusion uses a **quorum-grace timer**: once `minPanel` successful answers arrive, a short `stragglerGraceMs` timer begins. When it expires—or the `panelHardTimeoutMs` deadline hits—the collected answers proceed to judgment. This bounds worst-case latency while capturing fast responses immediately.

### Can the judge model stream its response or use tools?

Yes. Unlike panel calls, the **judge retains the original request's streaming flag and tool definitions**. This means the final synthesized answer can stream to clients or invoke tools if the original user request specified them.

### What happens if only one panel model succeeds?

Behavior depends on judge configuration. **Without an explicit `judgeModel`**, the single answer returns directly for minimal latency. **With an explicit judge configured**, that model still synthesizes a polished response even from one source—useful when quality guarantees outweigh speed.

### Where is the maximum panel size enforced?

The `maxPanel` limit (default 40) is validated early in `handleFusionChat` within [`open-sse/services/fusion.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/fusion.ts). Panels exceeding this threshold are rejected before any network calls begin, preventing memory exhaustion from oversized configurations.