How the FreeLLMAPI Fusion Panel Works: Multi-Model Synthesis Explained

The FreeLLMAPI Fusion panel works by routing requests with model: "fusion" to a multi-model synthesis pipeline that selects a diverse panel of backend LLMs, executes them in parallel, and combines their outputs using a judge model or a best-of strategy.

The Fusion panel is the core multi-model synthesis feature of the tashfeenahmed/freellmapi repository. When a client specifies the virtual model ID fusion, the service constructs a panel of distinct backend models, dispatches calls concurrently, and synthesizes a unified response. This architecture ensures robust, high-quality answers by leveraging multiple providers simultaneously rather than relying on a single model.

Triggering Fusion Mode

Fusion mode activates when the model field contains the literal string "fusion" or a fusion-prefixed variant. In server/src/services/fusion.ts, the isFusionModel function detects this trigger on lines 25-29, routing the request to the runFusion entry point.

{
  "model": "fusion",
  "messages": [{ "role": "user", "content": "Explain quantum computing" }]
}

Once triggered, the system constructs an effective configuration by merging dashboard-saved defaults with inline request parameters.

Configuration and Panel Construction

The Fusion configuration schema is defined by fusionConfigSchema in server/src/services/fusion.ts (lines 65-79). The resolveEffectiveConfig function (lines 122-148) validates and merges these settings to determine panel behavior.

Key configuration fields:

• models – Explicit list of model IDs to include (overrides auto-selection)
• k – Desired panel size, clamped to [1, fusion_max_k]
• judge – Model ID to act as the synthesizer
• strategy – "synthesize" (default) runs the judge; "best_of" returns the longest single answer
• expose_panel – Returns full per-model results in the x_fusion payload when true

Explicit vs Auto Panel Selection

The selectPanel function builds the candidate set using two distinct paths:

Explicit selection occurs when fusion.models is provided. Each ID is resolved via resolveFusionCandidate in server/src/services/router.ts (lines 61-84), with invalid or disabled IDs dropped and reported.

Auto selection draws from the fallback chain via getOrderedFusionChain (lines 607-609) in router.ts. The algorithm ensures platform diversity by selecting the first model from each distinct backend provider before filling remaining slots, preventing concentration on a single platform.

The final panel size is calculated as k = Math.min(Math.max(config.k ?? panelDefaultK(), 1), maxK) (line 76), returning three arrays: panel (active models), overflow (replacement candidates), and dropped (rejected IDs).

Dispatching Parallel Model Calls

Each panel slot is hard-pinned to a specific model via routePinnedModel in server/src/services/router.ts (lines 40-47). This guarantees that rate-limited or errored keys do not cause the slot to silently collapse onto a different model, preserving the intended diversity.

The runModelCall function (lines 95-123 in fusion.ts) executes each request by:

1. Rotating across the model's available keys
2. Applying rate-limit and cool-down checks
3. Logging via logRequest (line 11 in router.ts)
4. Recording outcomes via recordSuccess or recordRateLimitHit

If a slot fails, the system pulls replacements from the overflow array while maintaining platform diversity, limited to 2 × k total dispatches.

Judge Synthesis and Strategy

When the panel yields at least two successful answers (SYNTHESIS_QUORUM) and the strategy is "synthesize", the system initiates judge-based synthesis.

The buildJudgeMessages function (lines 408-426) constructs a prompt using JUDGE_SYSTEM_PROMPT (lines 401-406), concatenating the original conversation with anonymized panel responses.

The judge is invoked via:

• runJudgeStreaming (line 560) for streaming responses
• runModelCall for non-streaming requests

If the judge succeeds, its output becomes the final answer with synthesized: true. If the judge fails, the system falls back to the best-of strategy, selecting the longest successful panel answer.

Response Format and Diagnostics

The Fusion panel returns an OpenAI-compatible JSON structure with additional diagnostic fields:

{
  "model": "fusion",
  "choices": [{ "message": { "content": "Synthesized answer..." } }],
  "usage": { "prompt_tokens": 42, "completion_tokens": 123, "total_tokens": 165 },
  "_fusion": {
    "panel": [{ "platform": "groq", "model": "mixtral-8x7b-32768" }],
    "judge": "openrouter/gpt-4o-mini",
    "synthesized": true
  },
  "x_fusion": {
    "panel": [
      { "model": "groq/mixtral-8x7b-32768", "status": "ok", "content": "..." },
      { "model": "cerebras/llama-2-70b", "status": "ok", "content": "..." }
    ],
    "dropped": []
  }
}

The _fusion object (lines 986-1002) provides a lightweight routing summary including panel composition, judge model, and synthesis status. When expose_panel is true, the x_fusion object (lines 1005-1018) contains full per-model status, content, and error details.

Implementation Examples

Basic Fusion API Request

curl https://api.freellmapi.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fusion",
    "messages": [{"role": "user", "content": "Explain quantum tunnelling"}],
    "fusion": {
      "k": 4,
      "strategy": "synthesize",
      "judge": "openrouter/gpt-4o-mini",
      "expose_panel": true
    }
  }'

Streaming Fusion with JavaScript

const resp = await fetch('https://api.freellmapi.com/v1/chat/completions', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: 'fusion',
    stream: true,
    messages: [{ role: 'user', content: 'What are the health benefits of yoga?' }],
    fusion: { k: 3, judge: 'openrouter/gpt-4o-mini' }
  })
});

const reader = resp.body.getReader();
while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  const text = new TextDecoder().decode(value);
  // SSE frames contain _fusion panel updates and judge deltas
  console.log(text);
}

In streaming mode, the client receives incremental _fusion events for each panel answer (onPanel) followed by token-by-token judge output (onJudgeDelta).

Summary

• Virtual model trigger: Requests with "model": "fusion" activate the runFusion service in server/src/services/fusion.ts
• Diverse panel selection: Auto-selection ensures one model per platform via getOrderedFusionChain, while explicit selection uses resolveFusionCandidate
• Hard-pinned execution: routePinnedModel prevents slot collapse onto alternate models during rate limits
• Configurable synthesis: The "synthesize" strategy uses a judge model with SYNTHESIS_QUORUM requirement, while "best_of" returns the longest answer
• Rich diagnostics: The _fusion and x_fusion response fields expose panel composition, token usage, and per-model results

Frequently Asked Questions

What triggers the fusion panel in FreeLLMAPI?

The fusion panel triggers when the request body contains "model": "fusion" or a fusion-prefixed variant. The isFusionModel function in server/src/services/fusion.ts (lines 25-29) detects this identifier and routes the request to the runFusion handler, which initiates the multi-model synthesis pipeline.

How does the panel ensure model diversity?

The selectPanel function ensures diversity by using getOrderedFusionChain from server/src/services/router.ts to build a platform-diverse ordered list. The algorithm selects the first available model from each distinct backend provider (tracked via a seenPlatform set) before filling remaining slots, guaranteeing the panel spans multiple providers rather than clustering on one platform.

What happens if a model in the panel fails?

Failed panel slots trigger the overflow mechanism. The system pulls replacement models from the overflow array (maintained during initial panel selection) while respecting platform diversity constraints. Each slot is hard-pinned to its assigned model via routePinnedModel, ensuring that rate-limited or errored keys do not cause silent fallback to different models, with a maximum of 2 × k total dispatch attempts permitted.

How is the final answer synthesized?

When the strategy is "synthesize" and at least two panel models succeed (SYNTHESIS_QUORUM), the system constructs a judge prompt using buildJudgeMessages and JUDGE_SYSTEM_PROMPT (lines 401-426). The judge model (specified in judge config) evaluates anonymized panel responses and generates a unified answer. If the judge fails or the strategy is "best_of", the system returns the longest successful panel answer instead.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}