# How Fusion Multi-Model Synthesis Aggregates Responses in FreeLLM API

> Discover how Fusion multi-model synthesis in FreeLLM API aggregates responses using a panel of models and a judge model. Learn about this advanced workflow.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-08-31

---

**The Fusion feature implements a "panel-of-models + judge" workflow that aggregates responses by running multiple LLMs in parallel, then either selecting the best answer or synthesizing them through a dedicated judge model.**

The **fusion multi-model synthesis** feature in the `tashfeenahmed/freellmapi` repository transforms single requests into comprehensive answers by orchestrating diverse language models through a sophisticated aggregation pipeline. This system leverages parallel execution, intelligent fallback mechanisms, and optional judicial synthesis to produce higher-quality outputs than any single model could generate alone.

## Understanding the Fusion Architecture

At its core, Fusion operates on a **panel-of-models** principle. When a client sends a `fusion` request, the system does not route to one provider—instead, it distributes the prompt across multiple independent models and aggregates their outputs. The architecture consists of four primary stages: configuration resolution, panel construction, parallel execution, and answer synthesis.

The implementation lives primarily in [`server/src/services/fusion.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/fusion.ts), which orchestrates the entire pipeline from request validation to final response assembly. Supporting infrastructure in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) handles provider-specific routing, while [`server/src/lib/request-log.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/request-log.ts) ensures full observability through `fusion` tagged logging.

## Step-by-Step Response Aggregation Process

### Configuration Resolution

Before executing any model calls, Fusion determines the effective configuration through the `resolveEffectiveConfig` function in **fusion.ts** (lines 61-72). This merges the request's inline `fusion` object with dashboard-saved defaults to establish:

- The panel size `k` (number of models to consult)
- The specific panel models or auto-selection criteria
- The judge model identifier (if synthesis mode is enabled)
- The synthesis strategy (`synthesize` or `best_of`)

This resolution ensures that even minimal requests inherit sensible defaults while allowing granular per-request overrides.

### Panel Selection and Diversification

The `selectPanel` function (lines 49-86) constructs the execution panel through two distinct paths:

**Explicit Panel Mode:** When the request supplies a `models` array, each entry is validated for availability, tool/vision support, and deduplicated. Only valid, unique models enter the panel.

**Auto-Panel Construction:** Without explicit models, Fusion builds a diversified panel using `diversifyChain` (lines 17-36). This functionorders the fallback chain, filters for request-specific capabilities (tools, vision), and diversifies by **both** provider and model family to maximize perspective variety. The first *k* candidates become the active panel, while the next *k* form an **overflow queue** for refilling failed slots.

### Parallel Model Execution

Each panel member executes via `runModelCall` (lines 17-30), which handles the complexity of production LLM interactions:

- **Retry logic** with exponential backoff for transient failures
- **Rate-limit handling** through `isRateLimitSignal` checks from [`server/src/lib/error-classify.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/lib/error-classify.ts)
- **Provider cooldowns** to prevent cascading failures
- **Key rotation** across a model's available API keys

Failed slots automatically refill from the overflow queue, ensuring the target `okCount` of successful responses is met even when individual providers error out.

### Answer Collection Strategy

The `runFusion` function (lines 104-118) implements a **wave-based collection** system. Rather than awaiting all promises simultaneously, Fusion processes results in waves—first the initial *k* slots, then progressively drawing from the overflow list until sufficient successful answers accumulate.

Each completed slot reports through the optional `onPanel` hook, enabling real-time monitoring of panel performance. This wave approach optimizes latency by returning as soon as the minimum viable answer count is reached, without waiting for slow outliers.

### Final Answer Selection

Fusion applies a cascading decision tree to select or generate the final response:

**Tool-Call Shortcut:** If any panel model returns structured `tool_calls`, Fusion immediately selects the first such answer verbatim (lines 40-45). The judge is bypassed entirely, and the response wraps the tool call directly. This prioritizes actionable structured output over synthesized text.

**Best-of Strategy:** When `strategy: 'best_of'` is configured or fewer than two viable answers exist, Fusion returns the **longest panel answer** as a proxy for completeness (lines 99-103). This requires no judge invocation and minimizes latency for simple aggregation needs.

**Synthesize Mode (Default):** For standard synthesis, surviving panel texts feed into a **judge model**:

1. **Prompt Construction:** `buildJudgeMessages` (lines 99-118) assembles the judge prompt by injecting `JUDGE_SYSTEM_PROMPT` and formatting each panel answer as `--- Response N ---` within a user message appended to the original conversation context.

2. **Judge Execution:** The judge invokes either through `runJudgeStreaming` (for clients supplying `onJudgeDelta` hooks) or standard `runModelCall`. Streaming provides token-by-token delivery while regular calls return complete responses.

3. **Fallback Handling:** If the judge fails, the system automatically falls back to the best-of answer, ensuring robustness against judicial model outages.

## Implementation Details and Code Examples

The following TypeScript examples demonstrate practical usage patterns from [`server/src/services/fusion.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/fusion.ts):

```typescript
import { runFusion, FUSION_MODEL_ID } from './services/fusion.js';

// Simple fusion call with auto-panel selection and synthesis
await runFusion({
  messages: [{ role: 'user', content: 'Explain the theory of relativity.' }],
  config: {},                     // Empty config uses defaults
  options: { temperature: 0.7 },
  estimatedTokens: 256,
});

```

**Explicit panel configuration** with judicial synthesis and debug output:

```typescript
await runFusion({
  messages: [{ role: 'user', content: 'Summarise the last three news items.' }],
  config: {
    models: ['openrouter/google/gemini-1.5-flash', 'openrouter/anthropic/claude-3-opus-20240229'],
    k: 2,
    judge: 'openrouter/meta/llama-3.1-70b',
    expose_panel: true,          // Adds x_fusion field with full panel data
  },
  options: { temperature: 0.5 },
  estimatedTokens: 512,
});

```

**Best-of strategy** without judge invocation:

```typescript
await runFusion({
  messages: [{ role: 'user', content: 'Write a haiku about rain.' }],
  config: { strategy: 'best_of' },
  options: { temperature: 0.9 },
  estimatedTokens: 64,
});

```

The final response always includes a `_fusion` metadata field indicating which panel models participated and whether a judge was used. When `expose_panel: true` is set, the response contains `x_fusion` with the complete panel-wise payload for debugging.

## Summary

- **Configuration resolution** merges inline requests with dashboard defaults via `resolveEffectiveConfig` to establish panel parameters.
- **Panel selection** uses `diversifyChain` to maximize provider and model family diversity when auto-selecting models.
- **Parallel execution** through `runModelCall` implements robust retry logic, rate-limit handling, and automatic overflow refilling.
- **Wave-based collection** gathers answers progressively until `okCount` is satisfied, optimizing for latency.
- **Three-tier synthesis** prioritizes tool calls, falls back to longest-answer selection for `best_of` mode, or synthesizes via a judge model using `buildJudgeMessages`.
- **Observability** is built-in through `fusion` request-log tags and optional `onPanel`/`onJudgeDelta` hooks.

## Frequently Asked Questions

### What is the difference between `synthesize` and `best_of` strategies in Fusion?

The **synthesize** strategy (default) sends all panel responses to a dedicated judge model that generates a coherent combined answer using `buildJudgeMessages` and `JUDGE_SYSTEM_PROMPT`. The **best_of** strategy skips the judge entirely and returns the longest panel answer as a proxy for completeness, reducing latency and token costs when simple aggregation suffices.

### How does Fusion handle failures in individual panel models?

Fusion implements **overflow queue refilling** through the wave-based collection system in `runFusion`. When a panel slot fails, the system automatically substitutes the next candidate from the pre-calculated overflow list until the target `okCount` of successful responses is achieved. If the judge model fails during synthesis, Fusion falls back to the best-of answer to ensure request completion.

### Can I see individual model responses when using Fusion?

Yes. Set `expose_panel: true` in the fusion configuration to receive the `x_fusion` field in the response, which contains the full panel-wise payload including each model's individual output. The standard `_fusion` field always indicates which models participated and whether judicial synthesis was applied.

### What happens if a panel model returns a tool call instead of text?

Fusion immediately adopts the first tool call encountered among panel responses, bypassing both the judge and best-of selection logic. This **tool-call shortcut** (implemented in `runFusion` lines 40-45) returns the structured output verbatim, prioritizing actionable function calls over synthesized text responses.