How OmniRoute Executors Handle Upstream Requests: Base, Default, Cursor, Codex, and Vertex Explained

OmniRoute routes every LLM request through a hierarchy of executor classes—BaseExecutor, DefaultExecutor, CursorExecutor, CodexExecutor, and VertexExecutor—each overriding specific methods to build URLs, assemble headers, transform payloads, and manage retries for their respective providers.

Understanding how OmniRoute executors handle upstream requests is essential for troubleshooting provider-specific quirks or extending the router to new endpoints. The diegosouzapw/OmniRoute repository implements a strategy pattern where each executor inherits common transport logic from base.ts and selectively overrides behaviors to satisfy unique provider contracts—from OpenAI-compatible APIs to Google Vertex AI’s OAuth 2.0 requirements.

BaseExecutor: The Foundation

All executors inherit from BaseExecutor in open-sse/executors/base.ts. This class implements the generic lifecycle of an upstream request: URL resolution, header construction, payload transformation, retry logic, and credential refresh.

URL Resolution via buildUrl()

The buildUrl() method (lines 396–417) constructs the final endpoint. For OpenAI-compatible providers, it combines the custom base URL with an optional chat path. For other providers, it falls back to the static baseUrls array defined in the provider configuration.

// Conceptual flow inside BaseExecutor
protected buildUrl(config: ProviderConfig): string {
  if (config.isOpenAICompatible) {
    return `${config.baseUrl}${config.chatPath || '/v1/chat/completions'}`;
  }
  return config.baseUrls[0];
}

Header Construction Pipeline

Header generation occurs in two stages. First, buildHeadersPreamble() (lines 822–848) creates the baseline set including Content-Type, provider-specific headers, and an optional User-Agent from environment variables. It also resolves the effective API key, accounting for key rotation when multiple keys are configured.

Then, buildHeaders() (lines 1009–1031) finalizes the set by injecting the Authorization header, selecting the appropriate Accept header for streaming versus JSON responses, and normalizing Anthropic-specific header variations.

Request Transformation with transformRequest()

Before the fetch occurs, transformRequest() (lines 1033–1084) sanitizes the payload. It removes empty fields, fixes tool definitions, and strips provider-specific illegal keys. This ensures that downstream providers receive clean, valid JSON regardless of what the client originally sent.

Retry Logic and Credential Refresh

The shouldRetry() method (lines 1086–1090) implements circuit-breaker logic for rate limits. Using RETRY_CONFIG (max attempts = 2, delay = 2000 ms), it determines whether a 429 response should trigger a retry on the next URL in the provider’s fallback list.

For long-lived sessions, needsRefresh() (lines 1096–1115) checks the expiresAt timestamp against a provider-specific lead time. Subclasses override refreshCredentials() to silently obtain new tokens—critical for Vertex AI’s short-lived OAuth tokens.

DefaultExecutor: OpenAI-Compatible Providers

Located in open-sse/executors/default.ts, DefaultExecutor handles the majority of modern LLM APIs that follow the OpenAI specification (Anthropic-compatible endpoints, Groq, Mistral, etc.).

It inherits BaseExecutor wholesale and only overrides transformRequest() when the provider advertises a custom chatPath or requires specific body tweaks, such as explicitly adding the stream flag. If you are integrating a standard OpenAI-compatible provider, you are likely using this executor through the registry in open-sse/executors/index.ts.

CursorExecutor: Microsoft Cursor

The CursorExecutor in open-sse/executors/cursor.ts addresses Microsoft Cursor’s unique constraints:

• Custom Headers: Injects the mandatory x-cursor-model header required by the Cursor service.
• Payload Sanitization: Actively removes prompt_cache_retention from the request body (see the comment in BaseExecutor.transformRequest at lines 71–73) because Cursor rejects this field as unrecognized.
• Path Injection: For legacy endpoints, it may embed the model identifier directly into the URL path rather than the request body.

CodexExecutor: GitHub Codex

GitHub Codex requires specific handling found in open-sse/executors/codex.ts:

• Tool Model Prefix Stripping: Calls stripVersionedToolModelPrefix() (lines 5005–5016 in base.ts) to remove date-suffixes from tool model identifiers that Codex cannot parse.
• Authentication Variants: In addition to the standard “effective key” logic, it conditionally injects a Codex-API-Key header if the provider configuration supplies one, supporting GitHub’s dual-key authentication schemes.

VertexExecutor: Google Vertex AI

Google Vertex AI diverges significantly from the standard HTTP JSON pattern, necessitating the VertexExecutor in open-sse/executors/vertex.ts:

• URL Construction: Overrides buildUrl() to construct the Vertex-specific endpoint pattern: https://{location}-aiplatform.googleapis.com/v1/projects/{projectId}/locations/{location}/publishers/..., interpolating location and projectId from the provider configuration.
• OAuth 2.0 Headers: Uses Authorization: Bearer <accessToken> rather than API keys, requiring the credential refresh logic in BaseExecutor to fetch new tokens via Google’s identity endpoints.
• Body Wrapping: Wraps the standard chat payload in an instances array and moves generation parameters to a parameters object per Vertex’s REST contract.

How Requests Flow Through the System

When a client hits the OmniRoute API, the request traverses the following path:

1. Route Entry: src/app/api/v1/chat/completions/route.ts validates authentication and forwards the payload to open-sse/handlers/chatCore.ts.
2. Executor Selection: open-sse/executors/index.ts maps the provider ID (e.g., openai-compatible-…, cursor, codex, vertex) to the corresponding executor class.
3. Execution Orchestration: The executor’s execute() method (inherited from BaseExecutor) orchestrates the call:
   • Resolves base URL via resolveBaseUrl() → buildUrl().
   • Assembles headers via buildHeadersPreamble() → buildHeaders(), triggering credential refresh if needsRefresh() returns true.
   • Sanitizes the body via transformRequest().
   • Performs the fetch with timeout, abort-signal merging, and retry loops governed by shouldRetry().
4. Uniform Return: Returns a structured ExecutorExecuteResult containing { response, url, headers, transformedBody, transport }, allowing downstream services (MCP, A2A, streaming handlers) to process the response agnostically.

Code Examples

Standard OpenAI-Compatible Provider

import { getExecutor } from '@/open-sse/executors';

const exec = getExecutor('openai-compatible-groq', providerConfig);
const result = await exec.execute({
  model: 'llama-3.1-70b',
  stream: true,
  body: { messages: [{ role: 'user', content: 'Hello' }] },
  credentials: userCreds
});

Google Vertex AI Direct Usage

import { VertexExecutor } from '@/open-sse/executors/vertex';

const vertex = new VertexExecutor('vertex', vertexConfig);
const result = await vertex.execute({
  model: 'gemini-1.5-pro',
  stream: false,
  body: { 
    instances: [{ content: 'Write a haiku' }], 
    parameters: { temperature: 0.7 } 
  },
  credentials: vertexCreds
});

Cursor with Automatic Field Stripping

import { CursorExecutor } from '@/open-sse/executors/cursor';

const cursor = new CursorExecutor('cursor', cursorConfig);
// Note: `prompt_cache_retention` is automatically removed before sending upstream
const result = await cursor.execute({
  model: 'cursor-fast',
  stream: true,
  body: { 
    prompt: 'Explain recursion', 
    prompt_cache_retention: true 
  },
  credentials: cursorCreds
});

Summary

• BaseExecutor (open-sse/executors/base.ts) provides the shared infrastructure for URL building, header assembly, request transformation, retry logic, and credential refresh that all providers inherit.
• DefaultExecutor (open-sse/executors/default.ts) serves OpenAI-compatible providers with minimal overrides, covering the majority of LLM endpoints.
• CursorExecutor (open-sse/executors/cursor.ts) adds the x-cursor-model header and strips prompt_cache_retention to satisfy Microsoft Cursor’s strict payload validation.
• CodexExecutor (open-sse/executors/codex.ts) normalizes tool model names and supports GitHub’s dual-key authentication pattern.
• VertexExecutor (open-sse/executors/vertex.ts) handles Google Vertex AI’s OAuth 2.0 token refresh, custom URL patterns with location and projectId, and the instances/parameters body structure.
• The executor registry in open-sse/executors/index.ts maps provider IDs to classes, enabling the system to route requests through the correct strategy without client-side configuration changes.

Frequently Asked Questions

What is the difference between BaseExecutor and DefaultExecutor?

BaseExecutor is an abstract foundation containing provider-agnostic logic for HTTP transport, retries, and authentication. DefaultExecutor is a concrete implementation for OpenAI-compatible APIs that largely relies on the base class, only overriding transformRequest() when specific providers require minor body tweaks. Most providers in OmniRoute use DefaultExecutor unless they require custom headers, URL patterns, or authentication schemes like Cursor, Codex, or Vertex.

How does OmniRoute handle authentication for Google Vertex AI?

The VertexExecutor overrides credential handling to support OAuth 2.0 access tokens rather than static API keys. It calls needsRefresh() to check token expiration against a lead time, then executes refreshCredentials() to fetch a new Bearer token from Google’s identity endpoints before injecting it into the Authorization header via buildHeaders().

Why does CursorExecutor remove the prompt_cache_retention field?

Microsoft Cursor’s API strictly validates incoming payloads and rejects unrecognized keys. Because prompt_cache_retention is a provider-specific extension used by other services but invalid for Cursor, the CursorExecutor (via inherited logic in BaseExecutor.transformRequest) strips this field before serialization to prevent 400 Bad Request errors.

How does retry logic work across all executors?

All executors inherit shouldRetry() from BaseExecutor, which checks HTTP status codes (specifically 429) against the RETRY_CONFIG constant (max 2 attempts, 2000ms delay). If a rate limit is hit, the executor automatically attempts the request against the next URL in the provider’s baseUrls array, transparent to the calling code.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}