How OmniRoute’s Pipeline Combo Routing Strategy Works: Sequential LLM Chaining Explained

OmniRoute’s pipeline strategy chains multiple LLM models sequentially so that each model’s output becomes the next model’s input, creating a deterministic multi-stage processing flow implemented in open-sse/services/combo.ts.

The OmniRoute platform provides intelligent request routing across multiple AI providers. Among its 19 built-in combo routing strategies, the pipeline approach enables developers to create deterministic “assembly-line” workflows where one model preprocesses content before passing it to a more capable model for refinement. This article examines the implementation details, source code structure, and practical usage of the pipeline combo routing strategy in the diegosouzapw/OmniRoute repository.

What is the OmniRoute Pipeline Combo Routing Strategy?

OmniRoute’s routing layer supports combo strategies that dispatch a single request to multiple providers. The pipeline strategy specifically executes models in strict sequence, feeding the generated content from stage n into the input of stage n+1. This differs from parallel strategies like ensemble or fallback by creating a dependent chain where each step transforms the previous output.

According to the Auto-Combo design documentation in docs/routing/AUTO-COMBO.md, the pipeline strategy is selected either when a request explicitly specifies strategy: 'pipeline' or when the 14-factor scoring algorithm determines that sequential refinement optimizes for cost, latency, and quality constraints.

Implementation Details: How the Pipeline Executes

The core logic resides in open-sse/services/combo.ts, specifically within the handleCombo() function. When the router identifies a pipeline request, it executes the following sequence:

Target Resolution and Ordering

First, resolveComboTargets() builds an ordered list of model descriptors based on the request payload and routing policy. This function inspects the combo.targets array to determine the execution sequence, ensuring that authentication credentials and rate-limit quotas are validated for each provider before execution begins.

Sequential Model Execution Loop

The pipeline branch of handleCombo() loops over the resolved targets. For each target:

1. It invokes handleSingleModel(), which performs authentication, rate-limit checks, and executes the model via the appropriate executor (such as open-sse/executors/openaiExecutor.ts).
2. It captures the provider’s response text.
3. It injects that content into the next request body under messages[0].content, effectively treating the previous model’s output as the new user message.

Context Preservation Between Stages

The pipeline maintains the original system prompts and any static user messages, replacing only the assistant’s output at each stage. This ensures that every model in the chain sees the full conversation history up to that point, allowing later models to reference earlier context while applying their specific fine-tuning or capabilities.

Error Handling and Circuit Breakers

If any stage fails—whether due to a provider circuit breaker opening, an authentication error, or a model quota issue—the pipeline immediately aborts. The system invokes buildErrorBody() to generate a sanitized error response that respects global error-sanitization rules, preventing sensitive provider details from leaking to the client while clearly indicating which stage failed.

Response Aggregation and Return

After the final model completes execution, the accumulated result is wrapped in a standard chat-completion payload and returned to the client. The response preserves OpenAI-compatible formatting, ensuring compatibility with existing SDKs and client implementations.

Practical Implementation Examples

Direct API Usage

You can invoke the pipeline strategy directly via OmniRoute’s REST API by specifying the pipeline strategy and an ordered array of targets:

import fetch from 'node-fetch';

// Request a pipeline that first runs a fast model, then a more capable one
const body = {
  model: 'pipeline',
  combo: {
    strategy: 'pipeline',
    targets: [
      { provider: 'openai', model: 'gpt-3.5-turbo' },
      { provider: 'openai', model: 'gpt-4o' }
    ]
  },
  messages: [{ role: 'user', content: 'Summarize the attached article.' }]
};

const resp = await fetch('http://localhost:20128/v1/chat/completions', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify(body)
});

const result = await resp.json();
console.log(result.choices[0].message.content);

Programmatic Service Integration

For internal service calls, import the combo service directly:

import { handleCombo } from '@/open-sse/services/combo';
import { createRequestContext } from '@/open-sse/handlers/context';

const ctx = createRequestContext({
  model: 'pipeline',
  combo: { 
    strategy: 'pipeline', 
    targets: [
      { provider: 'anthropic', model: 'claude-3-haiku' },
      { provider: 'anthropic', model: 'claude-3-opus' }
    ] 
  },
  messages: [{ role: 'user', content: 'Translate this to French.' }]
});

const comboResult = await handleCombo(ctx);
console.log(comboResult.body.choices[0].message.content);

Architecture and Key Source Files

Understanding the pipeline implementation requires familiarity with these specific modules:

• open-sse/services/combo.ts – Contains handleCombo() and resolveComboTargets(), implementing the core pipeline loop and target resolution logic.
• open-sse/handlers/chatCore.ts – Entry point for chat completions; delegates to the combo layer when a combo strategy is detected in the request.
• open-sse/executors/openaiExecutor.ts – Executes individual model requests within the pipeline chain.
• docs/routing/AUTO-COMBO.md – Design document defining all 19 combo strategies, including pipeline selection criteria and 14-factor scoring rules.
• src/lib/db/comboMetrics.ts – Persists performance metrics for each pipeline execution, enabling the Auto-Combo optimizer to learn from historical pipeline performance.

Summary

• The pipeline combo routing strategy chains multiple LLMs sequentially, where each model’s output feeds into the next model’s input.
• Implementation resides primarily in open-sse/services/combo.ts, utilizing handleCombo() for orchestration and handleSingleModel() for individual stage execution.
• The strategy preserves full conversation context while replacing only the assistant’s content between stages.
• Circuit breakers and error sanitization via buildErrorBody() ensure that pipeline failures halt execution gracefully without exposing provider internals.
• Both REST API and programmatic TypeScript interfaces support explicit pipeline configuration through the combo.targets array.

Frequently Asked Questions

How does OmniRoute’s pipeline strategy differ from other combo strategies?

Unlike parallel strategies such as ensemble (which aggregates multiple simultaneous responses) or fallback (which tries alternatives until one succeeds), the pipeline strategy creates a deterministic dependency chain. Each stage must complete before the next begins, making it ideal for refinement workflows like preprocessing with a cheap model then polishing with an expensive one.

What happens if one model in the pipeline fails?

The pipeline implements fail-fast behavior. If any stage encounters an error—whether a network timeout, rate limit, or circuit breaker trigger—the entire chain aborts immediately. The system returns a sanitized error response generated by buildErrorBody() that indicates the failure stage without leaking sensitive provider credentials or internal stack traces.

Can I mix different providers in a single pipeline?

Yes. The targets array accepts mixed provider configurations, allowing you to chain models from OpenAI, Anthropic, Google, or any supported executor. Each target specifies its own provider and model fields, and handleSingleModel() routes each stage to the appropriate executor implementation (e.g., open-sse/executors/openaiExecutor.ts for OpenAI models).

Where is the pipeline strategy logic located in the source code?

The primary implementation lives in open-sse/services/combo.ts, specifically within the handleCombo() function’s pipeline branch. Target resolution occurs in resolveComboTargets(), while individual model execution delegates to handleSingleModel(). The design rationale and selection criteria are documented in docs/routing/AUTO-COMBO.md.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}