OmniRoute Routing Strategies: LKGp, Context-Relay, Fusion, and Headroom Compared

OmniRoute's four routing strategies determine how provider-model targets are ordered and processed—LKGp pins requests to the last successful provider, Context-Relay optimizes for conversation context size, Fusion synthesizes multiple model outputs through a judge panel, and Headroom selects connections with the most available capacity to prevent saturation.

OmniRoute is an open-source AI gateway that intelligently distributes requests across multiple LLM providers. Understanding the differences between its routing strategies is essential for optimizing latency, cost, and response quality in production workloads.

LKGp (Last Known Good Provider)

The LKGp strategy creates a deterministic fast-path by pinning requests to the provider that succeeded most recently for a specific combo.

In open-sse/services/combo/applyStrategyOrdering.ts (lines 46-85), when the strategy is set to "lkgp", the system imports getLKGP from the local database and queries for a record matching the combo name and ID. If a last-known-good provider exists, that target is moved to the front of the ordered list, ensuring subsequent requests hit the same endpoint until it fails.

if (strategy === "lkgp") {
  const { getLKGP } = await import("../../../src/lib/localDb");
  const lkgpProvider = await getLKGP(combo.name, combo.id || combo.name);
  // Target matching lkgpProvider is moved to index 0
}

Use LKGp when you have stable workloads and want to avoid the overhead of strategy calculation on every request, particularly when using a cheap, reliable provider that has proven healthy.

Context-Relay Strategy

The Context-Relay strategy prioritizes providers that can accommodate the full conversation context without truncation, making it ideal for multi-turn conversations.

According to the source code in open-sse/services/combo/applyStrategyOrdering.ts (lines 96-99), this strategy invokes sortTargetsByContextSize to reorder targets based on their maximum context window capacity. The strategy constant is defined in src/shared/constants/routingStrategies.ts (lines 10-15), where it exposes the UI label "contextRelay" and description keys for the dashboard.

else if (strategy === "context-optimized") {
  orderedTargets = sortTargetsByContextSize(orderedTargets);
}

Use Context-Relay for chat-based applications, code assistants, or long-form reasoning tasks where preserving the entire prompt history is critical for response quality.

Fusion Strategy (Panel + Judge)

The Fusion strategy implements a panel + judge pattern that distributes requests to multiple models in parallel and synthesizes their outputs into a single, higher-quality response.

Implemented in open-sse/services/fusion.ts (lines 4-38), the handleFusionChat routine executes three distinct phases:

1. Fan-out: Dispatches non-streaming calls to multiple panel models simultaneously
2. Collection: Gathers responses using a quorum-grace timer (defined in lines 46-88)
3. Synthesis: Constructs a judge prompt via buildJudgePrompt (lines 114-136) and forwards the aggregated context to a judge model for final output generation

export async function handleFusionChat({ /* ... */ }) {
  // 1️⃣ Fan-out panel calls to multiple providers
  // 2️⃣ Collect responses with quorum-grace timeout
  // 3️⃣ Build judge prompt and invoke judge model
}

Use Fusion when you need model diversity for complex tasks like summarization, multi-view reasoning, or when you require consensus quality exceeding what any single model can provide.

Headroom-Based Routing

The Headroom strategy performs capacity-aware load balancing by selecting the connection with the most available quota across your provider pool.

In open-sse/services/combo/quotaStrategies.ts (lines 79-87), the orderTargetsByHeadroom function expands each target to its possible connections and fetches 5-hour and weekly utilization metrics via getSaturation. It then ranks connections using the formula 1 - max(util5h, util7d), placing the connection with the lowest saturation (highest headroom) at the front of the list. The ranking logic resides in open-sse/services/combo/headroomRanking.ts.

export async function orderTargetsByHeadroom(
  targets, comboName, log, apiKeyAllowedConnectionIds
) {
  // Expand targets → fetch 5h & weekly saturation → rankByHeadroom(...)
}

Use Headroom for high-throughput workloads, batch jobs, or traffic spikes where you need to prevent provider saturation and maintain consistent response times across your connection pool.

Implementation Examples

Below are concrete API calls demonstrating how to configure each strategy when creating a combo in OmniRoute.

LKGp Configuration:

await fetch("/api/combo/create", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    name: "lkgp-combo",
    strategy: "lkgp",
    models: ["openai/gpt-4o-mini"], // Fallback if no LKGP record exists
  }),
});

Context-Relay Configuration:

await fetch("/api/combo/create", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    name: "relay-combo",
    strategy: "context-relay",
    models: ["anthropic/claude-3-sonnet", "openai/gpt-4o"],
  }),
});

Fusion Configuration:

await fetch("/api/combo/create", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    name: "fusion-combo",
    strategy: "fusion",
    models: ["openai/gpt-4o-mini", "anthropic/claude-3-opus"], // Panel
    config: {
      judgeModel: "openai/gpt-4o",
      fusionTuning: { minPanel: 2, stragglerGraceMs: 8000 },
    },
  }),
});

Headroom Configuration:

await fetch("/api/combo/create", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    name: "headroom-combo",
    strategy: "headroom",
    models: ["openai/gpt-4o-mini", "anthropic/claude-3-sonnet"],
  }),
});

Key Source Files

• open-sse/services/combo/applyStrategyOrdering.ts: Contains the strategy dispatcher and LKGP logic (lines 46-85), plus context-optimized sorting (lines 96-99)
• src/shared/constants/routingStrategies.ts: Defines strategy constants and UI metadata
• open-sse/services/fusion.ts: Implements the panel + judge orchestration (lines 4-38)
• open-sse/services/combo/quotaStrategies.ts: Houses orderTargetsByHeadroom and saturation metrics (lines 79-87)
• open-sse/services/combo/headroomRanking.ts: Provides the rankByHeadroom utility for capacity calculation

Summary

• LKGp creates a deterministic fast-path by pinning to the last successful provider, minimizing overhead for stable workloads
• Context-Relay sorts providers by context window size to prevent conversation truncation in multi-turn chats
• Fusion employs parallel panel execution with judge synthesis to maximize response quality through model diversity
• Headroom routes requests to the connection with the lowest utilization (calculated from 5-hour and weekly metrics) to ensure high availability

Frequently Asked Questions

When should I use LKGp instead of Headroom?

LKGp is optimal when you have a primary provider that is consistently reliable and you want to minimize routing decision latency. Headroom is superior when you run high-throughput workloads where any single connection might saturate, requiring dynamic load distribution across multiple providers.

How does the Fusion strategy handle partial failures in the panel?

According to the implementation in open-sse/services/fusion.ts, Fusion uses a quorum-grace timer during the collection phase. The system waits for a minimum number of panel responses (minPanel) up to a configured timeout (stragglerGraceMs). If some panel models fail or timeout, the judge synthesizes the available responses, ensuring the request succeeds as long as the quorum is met.

What metrics drive the Headroom routing calculation?

The Headroom strategy calculates available capacity using 5-hour and weekly utilization percentages retrieved via getSaturation in quotaStrategies.ts. It ranks connections by 1 - max(util5h, util7d), selecting the connection with the maximum headroom relative to its historical usage patterns.

Can I switch routing strategies dynamically for existing combos?

Yes. The strategy is stored as a property on the combo configuration and evaluated at request time in applyStrategyOrdering.ts. You can update a combo's strategy via the API without recreating the combo, and subsequent requests will immediately adopt the new routing logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}