Understanding the Request Pipeline in OmniRoute: Translator, Executor, and Response Transformation

OmniRoute's request pipeline converts incoming API calls into provider-specific formats, executes them via HTTP handlers, and transforms responses back to the client's expected schema using three distinct stages: translation, execution, and response transformation.

The diegosouzapw/OmniRoute repository implements a unified routing layer that normalizes interactions between OpenAI-compatible clients and diverse LLM providers. At the heart of this system lies a request pipeline that seamlessly handles format conversion, HTTP execution, and response transformation through a modular, three-stage architecture.

Stage 1: Request Translation

The pipeline begins in open-sse/translator/index.ts, where the translateRequest function parses incoming payloads and detects the source format. This module supports conversions from OpenAI-style requests into provider-specific schemas for Anthropic, Gemini, and other targets, handling tool calls, image data, and token limits during the transformation process.

Provider-Specific Translation Logic

Individual translator modules handle the mapping complexities for each provider pair. For example, open-sse/translator/request/openai-to-claude.ts and open-sse/translator/request/openai-to-gemini.ts manage the transformation of message structures, system prompts, and multimodal content into their respective native formats.

import { translateRequest } from '@/open-sse/translator';

// Convert OpenAI format to Gemini payload
const geminiPayload = translateRequest(
  openAiBody,
  'openai',    // source format
  'gemini'     // target format
);

Stage 2: Execution via the Executor Factory

After translation, the pipeline hands the normalized request to an executor selected by the factory function in open-sse/executors/index.ts. The getExecutor(providerId) function returns the appropriate handler for the target provider, such as the default executor defined in open-sse/executors/default.ts.

HTTP Handling and Streaming

The executor builds the final URL, headers, and body before performing the HTTP call to the upstream LLM provider. It implements retry logic with exponential backoff, handles streaming responses via Server-Sent Events (SSE), and manages abort signals for request cancellation.

import { getExecutor } from '@/open-sse/executors';

const executor = getExecutor('gemini');
const stream = await executor.execute(geminiPayload);

Stage 3: Response Transformation

Once the provider returns data, the pipeline reverses the transformation process. Response translators in open-sse/translator/response/openai-to-claude.ts and open-sse/translator/response/openai-to-gemini.ts convert provider-specific outputs back into the client's expected OpenAI-compatible format.

SSE Stream Transformation

For the Responses API, the system uses open-sse/transformer/responsesTransformer.ts to create a TransformStream that converts standard Chat Completion SSE chunks into the richer Responses API event format, enabling real-time transformation without buffering entire responses.

import { createResponsesApiTransformStream } from '@/open-sse/transformer/responsesTransformer';

const transformer = createResponsesApiTransformStream();
stream.pipeThrough(transformer).pipeTo(clientResponse);

End-to-End Pipeline Flow

The complete flow executes as follows:

1. A Next.js route (e.g., src/app/api/v1/chat/completions/route.ts) receives the incoming request
2. open-sse/handlers/chatCore.ts invokes translateRequest to reshape the payload according to the target provider's schema
3. The executor factory retrieves the correct executor via getExecutor and sends the HTTP request to the upstream provider
4. Raw responses pass through provider-specific response translators to normalize the data structure
5. For Responses API endpoints, the data streams through responsesTransformer.ts before reaching the client

Summary

• Request Translation: The translateRequest function in open-sse/translator/index.ts converts OpenAI-compatible requests into provider-specific formats using dedicated mapper files for each target like openai-to-claude.ts and openai-to-gemini.ts.
• Execution: The getExecutor factory in open-sse/executors/index.ts handles HTTP transport, retry logic with exponential backoff, and streaming for upstream LLM providers.
• Response Transformation: Provider-specific response translators and the responsesTransformer.ts stream converter normalize outputs back to client expectations, including real-time SSE conversion.
• Unified Interface: The pipeline enables seamless routing through a single endpoint regardless of the underlying LLM provider's native API format.

Frequently Asked Questions

How does OmniRoute handle different API formats between providers?

OmniRoute uses dedicated translator modules located in open-sse/translator/request/ and open-sse/translator/response/ to map between OpenAI-compatible schemas and provider-specific formats like Anthropic's Claude or Google's Gemini. The translateRequest function automatically detects source and target formats to apply the correct transformation logic for messages, tool calls, and multimodal content.

What happens if an LLM provider request fails during execution?

The executor implements robust error handling including retry mechanisms with exponential backoff, proper abort signal cleanup, and streaming error management. These features are centralized in the default executor at open-sse/executors/default.ts and accessed through the factory pattern in open-sse/executors/index.ts, ensuring failed requests are retried according to configured policies before returning an error to the client.

Can OmniRoute convert streaming responses in real-time?

Yes. The pipeline supports Server-Sent Events (SSE) conversion through open-sse/transformer/responsesTransformer.ts, which creates a TransformStream that converts Chat Completion SSE chunks into Responses API format on the fly. This enables real-time transformation without buffering entire responses, maintaining low latency for streaming completions.

Where does the request pipeline entry point reside in the codebase?

The pipeline initiates in Next.js API routes such as src/app/api/v1/chat/completions/route.ts, which delegates to open-sse/handlers/chatCore.ts to orchestrate the translation, execution, and response transformation stages. This handler coordinates the three-stage pipeline by invoking the translator, executor factory, and response handlers in sequence.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}