How to Create Multi-Model Fallback Combos with OmniRoute's Combo Builder

OmniRoute enables resilient AI routing by letting you define ordered fallback chains of models that automatically retry requests when the primary provider fails, without exposing the complexity to your end users.

Multi-model fallback combos are a core feature of the OmniRoute routing engine that allows you to treat multiple AI providers as a single logical endpoint. By configuring these combos through the VSCode-token-protected API, you can ensure high availability for critical workloads. This guide walks through creating, persisting, and executing fallback chains using the actual source implementation in the diegosouzapw/OmniRoute repository.

What Are Multi-Model Fallback Combos?

A combo is an ordered collection of model configurations stored as a single named entity. When you send a request using a combo name as the model identifier, OmniRoute's open-sse/services/comboResolver.ts expands that name into a concrete fallback list. The system then attempts each model in sequence until one succeeds or the chain is exhausted.

The combo definition lives in the SQLite-backed combos table, managed through src/lib/db/combos.ts. Each entry contains an id, name, and definition_json field that stores the ordered array of target models and execution strategy.

Creating a Combo via the VSCode Token API

Combos are created through a protected API route that requires a valid VSCode token. The endpoint is implemented in src/app/api/v1/vscode/combos/[token]/[[...slug]]/route.ts, which validates the payload using Zod and persists it via the database layer.

Send a POST request with your combo definition:

await fetch(
  `http://localhost/api/v1/vscode/combos/${encodeURIComponent(token)}/api/create`,
  {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      name: 'my-fallback-combo',
      // Ordered list of model IDs; first is primary, later are fallbacks
      targets: [
        { provider: 'openai', model: 'gpt-4o-mini' },
        { provider: 'anthropic', model: 'claude-3.5-sonnet' },
        { provider: 'gemini', model: 'gemini-1.5-flash' },
      ],
      // Optional: strategy = 'strict' | 'weighted' | 'auto-fallback'
      strategy: 'strict',
    }),
  },
);

The database abstraction in src/lib/db/combos.ts provides the createCombo function that inserts this data into the combos table, making it immediately available for resolution.

How the Fallback Chain Executes

When a request arrives at /v1/chat/completions with a combo name as the model parameter, OmniRoute executes a multi-stage resolution process:

1. Combo Resolution: open-sse/services/comboResolver.ts calls resolveComboTargets() to look up the combo by name and expand it into a list of concrete model IDs.
2. Execution Loop: open-sse/handlers/chatCore.ts attempts the request against the first model in the list.
3. Error Classification: If the response triggers a fallback-eligible error, the handler automatically retries with the next model in the chain.
4. Client Response: The caller receives only the first successful response; the retry chain remains invisible.

Fallback Eligibility Detection

Not all errors trigger a fallback. According to the implementation in open-sse/services/accountFallback.ts, the checkFallbackError() function specifically identifies transient upstream failures. A retry occurs only for:

• HTTP 408 (Request Timeout)
• HTTP 500–504 (Server errors)
• Provider circuit-breaker open states
• Quota-exceeded errors

Permanent failures like 401 (Unauthorized) or 403 (Forbidden) halt execution immediately and return the error to the client, preventing unnecessary token consumption on invalid credentials.

Combo Resolution Strategy

The definition_json field supports multiple execution strategies interpreted by the combo resolver:

• strict: Process targets in order, stopping at the first success.
• weighted: Distribute load according to per-model weights while maintaining fallback capability.
• auto-fallback: Dynamically select targets based on current provider health metrics.

Using a Combo in API Requests

Once persisted, reference the combo by its name in any chat completion request:

await fetch('http://localhost/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: `Bearer ${apiKey}`,
  },
  body: JSON.stringify({
    model: 'my-fallback-combo', // Resolves to the fallback list
    messages: [{ role: 'user', content: 'Explain quantum tunnelling.' }],
  }),
});

The router transparently handles the complexity of trying OpenAI first, falling back to Anthropic if that fails, and finally attempting Gemini if necessary.

Inspecting and Managing Combos

For administrative purposes, you can retrieve combo definitions directly from the database using helper functions in src/lib/db/combos.ts:

import { getDbInstance } from '@/src/lib/db/core.ts';
import { combosDb } from '@/src/lib/db/combos.ts';

const db = await getDbInstance();
const combo = await combosDb.getComboByName('my-fallback-combo');
console.log('Combo definition:', combo?.definition_json);

The database layer also exposes listCombos() for enumeration and deleteCombo() for cleanup. All combo mutations are version-controlled via migrations in src/lib/db/migrations/, ensuring schema consistency across deployments.

Summary

• Multi-model fallback combos in OmniRoute act as resilient, named aliases for ordered chains of AI providers.
• Create combos via the VSCode-token-protected API at src/app/api/v1/vscode/combos/[token]/[[...slug]]/route.ts with strict, weighted, or auto-fallback strategies.
• The combo resolver (open-sse/services/comboResolver.ts) expands combo names into concrete target lists at request time.
• Fallback eligibility is determined by checkFallbackError() in open-sse/services/accountFallback.ts, which only retries transient errors (408, 500-504, circuit-open, quota-exceeded).
• The executor loop in open-sse/handlers/chatCore.ts handles automatic retries without exposing the chain complexity to API consumers.

Frequently Asked Questions

What happens if all models in a combo fail?

If every model in the fallback chain returns an error (or a non-fallback-eligible error occurs), OmniRoute returns the final error response to the client. The request fails only after exhausting all configured alternatives, ensuring you only receive errors when no provider can satisfy the request.

Can I update an existing combo without changing my API calls?

Yes. Since combos are resolved by name at request time, updating the definition_json in the combos table via the VSCode token API immediately affects all subsequent requests using that combo name. The src/lib/db/combos.ts module handles updates through the same route handlers used for creation.

How does OmniRoute handle streaming responses with combos?

The executor loop in open-sse/handlers/chatCore.ts manages streaming by attempting the stream against the primary model. If a fallback-eligible error occurs mid-stream, the connection is transparently switched to the next model in the combo chain, maintaining the streaming interface to the client without interruption.

Is there a limit to how many models I can chain in a single combo?

While the schema in src/lib/db/combos.ts stores the target list as JSON without a hardcoded length limit, practical limits depend on your latency requirements and timeout configurations. Each additional fallback adds potential latency, so most production combos use 2-3 models for optimal reliability-to-latency ratios.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}