How to Troubleshoot Common OmniRoute Errors: A Complete Guide to Debugging LLM Proxy Issues

Troubleshoot OmniRoute errors by tracing them through the validation, authentication, provider dispatch, and SSE streaming layers, using the error sanitization utilities in open-sse/utils/error.ts and provider classification logic in src/sse/services/auth.ts to isolate root causes.

OmniRoute is an open-source LLM proxy and router that normalizes requests across multiple providers. When errors occur—ranging from 400 Bad Request to 503 Service Unavailable—they propagate through a specific pipeline involving Zod validation, credential checks, rate limiting, and upstream provider handling. Understanding this architecture allows you to pinpoint whether an error originates from client input, authentication, policy enforcement, or the downstream LLM provider.

OmniRoute Error Handling Architecture

Errors in OmniRoute surface through a layered pipeline that processes every request:

Client → /v1/chat/completions → CORS → Zod Validation → Authentication
  → Policy & Prompt Injection Guard → Provider Selection → Circuit Breaker Check
  → Upstream Request → SSE Stream/JSON Response → Error Sanitization

The primary error generation points include:

• src/sse/handlers/chat.ts – Entry point for body parsing and initial validation
• src/sse/handlers/chatHelpers.ts – Core orchestration and stream error handling
• src/sse/services/auth.ts – Credential validation and provider error classification
• open-sse/utils/error.ts – Standardized error response construction
• open-sse/utils/errorSanitization.ts – Stack trace and PII removal

All errors pass through sanitizeErrorMessage before reaching the client to prevent leaking sensitive implementation details.

Common OmniRoute Error Categories and Solutions

Validation Errors (400 Bad Request)

Symptoms: "Invalid JSON body", "messages: Expected array", or "Missing model" responses.

These errors originate from Zod schema validation in the API route handler. When the request body fails validation—whether from malformed JSON, missing required fields like model or messages, or incorrect data types—OmniRoute returns a 400 status.

Troubleshooting steps:

1. Inspect the request payload against the schemas defined in src/sse/handlers/chat.ts
2. Verify that the model string matches the regex patterns in src/sse/handlers/chatHelpers.ts (typically ^[a-z0-9_-]+$)
3. Use curl -v or browser DevTools to capture the exact payload being sent

Authentication Errors (401/403)

Symptoms: "Invalid API key", "OAuth token expired", or "Forbidden" responses.

The validateApiKey function in src/sse/services/auth.ts verifies credentials against the database. Failures occur when the Authorization header is missing, malformed, or references revoked keys.

Troubleshooting steps:

1. Verify the API key format matches the expected prefix and length
2. Check that environment variables in .env are correctly loaded for database connections src/lib/db/credential.ts
3. Inspect the connection.lastError property in logs for the raw upstream authentication failure before sanitization

Rate-Limiting and Quota Errors (429 Too Many Requests)

Symptoms: "Rate limited", "Quota exhausted", or retry-after headers in responses.

OmniRoute implements per-account and per-provider rate limiting through src/sse/services/rateLimitManager/. When limits are exceeded, the system stores a rateLimitedUntil timestamp on the connection object.

Troubleshooting steps:

1. Search logs for log.warn("CHAT", …) entries containing retry-after information
2. Check the rateLimitedUntil timestamp on the connection object in src/sse/services/auth.ts
3. Adjust rate limit configurations in feature flags if necessary src/shared/constants/featureFlagDefinitions.ts

Provider-Level Errors (500-504)

Symptoms: "Upstream service error", "Model not found", or "Provider request failed".

These indicate downstream LLM service failures. The classifyProviderError function in src/sse/services/auth.ts categorizes HTTP status codes from upstream providers and maps them to user-friendly messages.

Troubleshooting steps:

1. Check connection.lastError for the original HTTP status and error body from the upstream provider
2. Verify the requested model exists in the provider's catalog
3. Review the sanitized error message through open-sse/utils/errorSanitization.ts to ensure you're seeing the safe version of the error

Circuit-Breaker and Cooldown Errors (503 Service Unavailable)

Symptoms: "All accounts cooling down", "Provider temporarily blocked", or persistent 503 statuses.

When upstream failures exceed thresholds defined in open-sse/config/constants.ts (PROVIDER_PROFILES), the circuit breaker opens to prevent cascading failures.

Troubleshooting steps:

1. Query the /api/monitoring/health/route.ts endpoint to check provider health states
2. Review the circuitBreakerReset timeout values in open-sse/config/constants.ts
3. Wait for the automatic reset interval or manually reset the provider state in the database if you've confirmed the upstream service is healthy

SSE Streaming Errors (500, Early EOF)

Symptoms: "Stream early EOF", "Unexpected end of SSE", or truncated streaming responses.

These occur when network interruptions or malformed delta chunks break the Server-Sent Events connection. src/sse/handlers/chatHelpers.ts implements retry logic for transient stream failures.

Troubleshooting steps:

1. Check the STREAM_EARLY_EOF_MAX_RETRIES constant in src/sse/handlers/chatHelpers.ts
2. Search logs for log.error("CHAT", …) messages containing the failing chunk content
3. Verify upstream provider streaming capabilities and network stability between OmniRoute and the LLM service

Code Examples for Error Handling

Building Standardized Error Responses

Use the errorResponse helper to maintain consistent error formatting across the application:

import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
import { HTTP_STATUS } from "@omniroute/shared/constants/http.ts";

function rejectInvalidModel(model: string) {
  const message = `Invalid model format: ${model}. Expected pattern: ^[a-z0-9_-]+$`;
  return { 
    error: errorResponse(HTTP_STATUS.BAD_REQUEST, message),
    status: 400 
  };
}

Source: Pattern derived from src/sse/handlers/chatHelpers.ts

Sanitizing Upstream Errors

Prevent data leakage by sanitizing raw provider errors before sending them to clients:

import { sanitizeErrorMessage } from "@omniroute/open-sse/utils/errorSanitization.ts";
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";

function handleProviderFailure(status: number, rawError: string) {
  // Removes stack traces and PII
  const safeMessage = sanitizeErrorMessage(rawError) 
    || "Provider request failed";
    
  return { 
    error: errorResponse(status, safeMessage) 
  };
}

Source: Pattern derived from src/sse/services/auth.ts

Logging Provider Failures

Implement consistent logging to capture error context for debugging:

log.error(
  "CHAT",
  `❌ Provider ${provider} [${status}]: ${errorMsg}. ` +
  `Account: ${accountId}, Model: ${model}, ` +
  `LastError: ${sanitizeErrorMessage(connection.lastError)}`
);

Source: Pattern derived from src/sse/services/auth.ts

Summary

• Validate input first: Check Zod schemas in chat.ts and regex patterns in chatHelpers.ts for 400 errors
• Verify authentication: Inspect validateApiKey in auth.ts and credential database entries for 401/403 errors
• Monitor rate limits: Review rateLimitManager directory and rateLimitedUntil timestamps for 429 errors
• Classify provider issues: Use classifyProviderError in auth.ts and check connection.lastError for 500-504 errors
• Check circuit state: Examine PROVIDER_PROFILES in constants.ts and health endpoints for 503 errors
• Debug streams: Review STREAM_EARLY_EOF_MAX_RETRIES and SSE handlers in chatHelpers.ts for streaming failures

Frequently Asked Questions

How do I fix "Provider request failed" errors in OmniRoute?

Check the connection.lastError property in src/sse/services/auth.ts to view the unsanitized upstream error. Verify the model name is correct, the provider's circuit breaker is not open (check health endpoints), and that the account has available quota in the rate limit manager.

Where does OmniRoute sanitize error messages to prevent data leaks?

Error sanitization occurs in open-sse/utils/errorSanitization.ts, which removes stack traces, internal paths, and personally identifiable information from upstream errors before they reach the client through errorResponse in open-sse/utils/error.ts.

What causes "All accounts cooling down" 503 errors in OmniRoute?

This indicates the circuit breaker has opened for that provider due to repeated upstream failures. The thresholds are configured in open-sse/config/constants.ts within PROVIDER_PROFILES. The error persists until the circuitBreakerReset timeout expires or the provider health check passes.

How can I debug SSE stream early EOF errors?

Early EOF errors indicate the upstream provider closed the connection prematurely. Check the retry counter against STREAM_EARLY_EOF_MAX_RETRIES in src/sse/handlers/chatHelpers.ts. Enable verbose logging to capture the specific chunk that caused the failure, then verify network stability and the upstream provider's streaming implementation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}