What Is `open-sse/handlers/chatCore.ts` in OmniRoute? The Central Chat Request Dispatcher Explained

open-sse/handlers/chatCore.ts exports the handleChatCore function, which serves as the single orchestration point for every AI completion request in OmniRoute, managing the entire lifecycle from authentication and routing to streaming pipelines and telemetry.

OmniRoute is an open-source AI gateway that standardizes requests across multiple providers. At its core lies open-sse/handlers/chatCore.ts, a TypeScript module implementing the handleChatCore dispatcher that all chat endpoints invoke. This file unifies authentication, validation, model routing, and error normalization into a ten-stage pipeline, ensuring consistent behavior whether clients connect via Server-Sent Events (SSE) or standard HTTP workers.

Core Responsibilities of the handleChatCore Function

The handleChatCore function acts as a unified abstraction layer. It accepts a structured configuration object containing the HTTP body, model metadata, credentials, and telemetry callbacks, then executes a rigorous pipeline before returning a standardized result contract.

Entry Point and Telemetry Initialization

Upon invocation, handleChatCore generates a request-wide trace ID and immediately logs a request.started event. It establishes a stage-trace helper for fine-grained debugging, ensuring every subsequent operation can be correlated across distributed logs.

Pre-Flight Guards and Validation

Before processing begins, the function runs a resource-pressure guard to check system capacity, performs an idempotency lookup to prevent duplicate processing, and executes optional plugin-on-request filters. If any guard fails—such as a plugin returning a 403—the function exits early with a structured error via buildErrorBody.

System Prompt Injection

The dispatcher automatically injects a global system prompt into the request body. If user-configured custom prompts are enabled in cached settings, these are merged into the message array at this stage, ensuring consistent AI behavior across all providers.

Device and Connection Tracking

For analytics and quota enforcement, handleChatCore records the client’s IP address and User-Agent string against the API key metadata. This enables per-key analytics and helps track device-specific usage patterns for rate limiting.

Routing and Model Resolution

This stage resolves the request format, target provider format, and any combo-routing metadata. It handles background-task redirects, applies model-lifecycle policies such as deprecation warnings, and strips Claude-specific effort variants when routing to non-Anthropic providers.

Tool Handling and Fallbacks

The function normalizes OpenAI-compatible tool definitions. When encountering unsupported native tools like web_search or web_fetch, it converts them into OmniRoute fallback bodies and enforces tool-calling compatibility checks against the target model’s capabilities.

Compression and Usage Accounting

If the request body exceeds size thresholds, the dispatcher determines whether compression is required and attaches a compression-usage receipt after analytics processing. It updates internal usage buffers to track token consumption preemptively.

Header Preparation and Protocol Detection

handleChatCore constructs upstream request headers, including any custom per-connection headers passed by the client. It also detects whether the downstream client expects streaming (SSE) or JSON responses to determine the appropriate output pipeline.

Stream and Non-Stream Pipeline Assembly

For streaming requests, the function calls assembleStreamingPipeline, which configures heartbeat transforms to keep connections alive, enforces token budget limits, and attaches finalization logic. For non-streaming requests, it prepares a synchronous JSON response structure instead.

Error Handling and Retry Logic

The upstream executor call is wrapped with robust retry logic that classifies provider-specific errors. The dispatcher records key-health status for circuit-breaker patterns and creates uniform error bodies using buildErrorBody from open-sse/utils/error.ts, ensuring clients receive predictable error shapes regardless of the upstream provider.

Post-Call Telemetry and Guardrails

After the upstream response completes, handleChatCore emits routing events, records final usage and cost metrics, and fires post-call guardrail hooks from the src/lib/guardrails registry. It logs the outcome to the live dashboard before returning control to the caller.

How SSE and Worker Handlers Consume handleChatCore

Rather than duplicating logic, OmniRoute’s transport-specific handlers act as thin wrappers around handleChatCore. Both the SSE endpoint and the Worker endpoint import the dispatcher and pass transport-specific callbacks.

SSE Handler Implementation (chat.ts)

The open-sse/handlers/chat.ts file handles Server-Sent Events. It deserializes the request, extracts connection metadata, and invokes handleChatCore:

import { handleChatCore } from "./chatCore.ts";

export async function GET(request) {
  const body = await request.json();
  const modelInfo = { provider: "openai", model: body.model };
  const credentials = await getProviderCredentials(...);
  const log = await createRequestLogger(...);

  const result = await handleChatCore({
    body,
    modelInfo,
    credentials,
    log,
    onCredentialsRefreshed: () => {},
    onRequestSuccess: () => {},
    clientRawRequest: request,
    connectionId: request.headers.get("x-connection-id"),
    apiKeyInfo: await getApiKeyInfo(request),
    userAgent: request.headers.get("user-agent"),
  });

  if (result.success && result.response?.stream) {
    return new Response(result.response.body, { headers: result.response.headers });
  }
  return new Response(JSON.stringify(result.response), { status: result.status });
}

Worker Handler Implementation (chatWorker.ts)

For non-SSE HTTP requests, open-sse/handlers/chatWorker.ts follows an identical pattern but expects a JSON payload rather than a stream:

import { handleChatCore } from "./chatCore.ts";

export async function POST(request) {
  const body = await request.json();
  // Setup similar to SSE handler...
  
  const result = await handleChatCore({ /* options */ });

  // Workers expect plain JSON
  return new Response(JSON.stringify(result.response), { status: result.status });
}

Return Contract and Consistency

Regardless of the transport layer, handleChatCore always resolves to a standardized object:

{
  success: boolean,
  response: any,
  status: number,
  error: Error | null
}

This contract allows both the SSE handler and Worker to uniformly handle success states, fallbacks, or failures without transport-specific branching logic in the core dispatcher.

Key Files in the OmniRoute Processing Pipeline

While chatCore.ts orchestrates the flow, it delegates specialized tasks to dedicated services:

• open-sse/handlers/chat.ts – SSE transport wrapper that forwards to handleChatCore.
• open-sse/handlers/chatWorker.ts – HTTP (non-SSE) wrapper also using handleChatCore.
• open-sse/services/routing/index.ts – Creates routing events consumed by the telemetry system.
• open-sse/utils/error.ts – Provides buildErrorBody for uniform error formatting.
• src/lib/guardrails – Registry of post-call guardrail hooks invoked after completion.
• src/lib/usage – Tracks token usage, cost calculation, and quota enforcement.
• open-sse/services/compression – Handles request-body compression and analytics.

Summary

• open-sse/handlers/chatCore.ts implements the handleChatCore function, the single entry point for all AI completion requests in OmniRoute.
• It executes a ten-stage pipeline covering validation, prompt injection, routing, tool normalization, compression, and telemetry.
• The function provides a unified return contract ({ success, response, status, error }) that standardizes error handling across SSE and HTTP transports.
• Specialized logic is delegated to dedicated services like open-sse/services/routing and src/lib/guardrails, keeping the core dispatcher focused on orchestration.
• Both open-sse/handlers/chat.ts and open-sse/handlers/chatWorker.ts act as thin wrappers, delegating all business logic to handleChatCore.

Frequently Asked Questions

What makes handleChatCore different from the regular chat.ts handler?

chat.ts is a transport-specific wrapper that handles Server-Sent Events (SSE) protocol details like connection headers and stream formatting. handleChatCore contains the actual business logic—routing, authentication, retry logic, and telemetry—that applies regardless of transport. According to the OmniRoute source code, all transport handlers delegate to handleChatCore to ensure consistent behavior across the entire codebase.

How does chatCore.ts handle errors from upstream AI providers?

The dispatcher wraps upstream calls with retry logic and classifies provider-specific errors into standardized categories. It uses the buildErrorBody utility from open-sse/utils/error.ts to create uniform error responses. This ensures clients receive consistent error shapes (status codes and message formats) even when underlying providers like OpenAI, Anthropic, or Google return different error structures.

Can handleChatCore process both streaming and non-streaming requests?

Yes. The function detects the client's expected response format during the header preparation phase. For streaming requests, it assembles a pipeline via assembleStreamingPipeline that includes heartbeat transforms and token budget enforcement. For non-streaming requests, it constructs a synchronous JSON response. Both paths return the same standardized contract to the caller.

Where does chatCore.ts fit in OmniRoute's request lifecycle?

It sits at the center of the request lifecycle. After initial HTTP parsing by transport handlers (SSE or Worker), handleChatCore takes over to perform validation, routing, tool handling, and telemetry. It invokes downstream services like src/lib/usage for accounting and src/lib/guardrails for post-call safety checks, making it the orchestration hub that coordinates all subsequent processing stages.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}