Does OmniRoute Support Real-Time Traffic Data in Routing Calculations?

OmniRoute does not incorporate real-time traffic metrics into its routing decisions; instead, it provides a Traffic-Inspector toolset for observability and monitoring.

OmniRoute, an open-source routing gateway hosted at diegosouzapw/OmniRoute, handles AI model selection through deterministic rules rather than dynamic traffic conditions. While real-time traffic data is captured and streamed for operational visibility, the core routing engine remains decoupled from live metrics, relying solely on static configuration and provider metadata.

How the Combo Routing Engine Processes Requests

The routing logic in OmniRoute is centralized in the combo engine located at open-sse/services/combo.ts. This module evaluates provider capabilities, cost settings, quota consumption, and configured routing strategies to determine request distribution.

Static Decision Making Without Live Metrics

According to the source code analysis, the combo engine does not ingest live traffic-volume or latency metrics into its calculation algorithm. All routing decisions are deterministic, based on pre-defined combo definitions and static provider configurations. This architectural choice ensures predictable behavior and eliminates latency spikes that could occur from real-time data processing during the routing phase.

Real-Time Traffic Observability Architecture

Although OmniRoute excludes real-time data from routing calculations, it implements a comprehensive Traffic-Inspector infrastructure for monitoring and debugging. This system captures every request and response, publishing them to a real-time event bus without impacting the routing path.

WebSocket Streaming Endpoint

The Traffic-Inspector exposes a WebSocket endpoint at src/app/api/tools/traffic-inspector/ws/route.ts that streams WsEvent frames to connected dashboards. This enables live visualization of traffic patterns, error rates, and usage statistics as they occur.

In-Memory Ring Buffer

Request and response data is temporarily stored in an in-memory ring buffer implemented in src/lib/proxyLogger.ts. This buffer maintains a short history of recent transactions, allowing operators to inspect recent activity without persistent storage overhead.

Typed Event Bus

The event publishing system relies on src/lib/events/eventBus.ts, a typed EventEmitter that decouples the inspector from dashboard consumers. This ensures that real-time monitoring infrastructure operates independently of the critical routing path.

Accessing Real-Time Traffic Data

OmniRoute provides multiple interfaces for consuming real-time traffic data through its Traffic-Inspector toolset.

Dashboard Integration with React Hooks

The src/hooks/useLiveDashboard.ts hook provides a React interface for WebSocket connectivity:

// React component in the dashboard
import { useLiveDashboard } from '@/hooks/useLiveDashboard';

export default function LiveTraffic() {
  const { events, connect, disconnect } = useLiveDashboard();

  useEffect(() => {
    connect();               // opens WS to /api/tools/traffic-inspector/ws
    return () => disconnect();
  }, []);

  return (
    <ul>
      {events.map((e) => (
        <li key={e.id}>
          {e.type}: {e.payload?.method} {e.payload?.path}
        </li>
      ))}
    </ul>
  );
}

Ring Buffer Inspection via Proxy Logger

For programmatic access to recent traffic, the proxy logger exposes a snapshot method:

import { getProxyLogSnapshot } from '@/lib/proxyLogger';

// Grab the last N request/response records
const snapshot = getProxyLogSnapshot({ limit: 100 });
snapshot.forEach((entry) => {
  console.log(`[${entry.id}] ${entry.method} ${entry.path} → ${entry.status}`);
});

REST API Endpoints

Traffic data is also accessible through REST endpoints defined in src/app/api/tools/traffic-inspector/**/route.ts:

# List recent traffic-inspector sessions

curl -H "Authorization: Bearer $OMNIRoute_API_KEY" \
     https://localhost/api/tools/traffic-inspector/sessions

# Retrieve detailed request logs for a session

curl -H "Authorization: Bearer $OMNIRoute_API_KEY" \
     https://localhost/api/tools/traffic-inspector/sessions/<session-id>/requests

Feature Flags and Configuration

Real-time dashboard functionality is controlled by the LIVE_DASHBOARD feature flag defined in src/shared/constants/featureFlagDefinitions.ts. This toggle enables or disables the WebSocket UI components without affecting the underlying routing logic, ensuring that observability features can be safely enabled or disabled in production environments.

Summary

• OmniRoute does not factor real-time traffic data into routing decisions, maintaining deterministic behavior in the combo engine (open-sse/services/combo.ts).
• The Traffic-Inspector toolset provides comprehensive observability through WebSocket streams, REST APIs, and in-memory buffers.
• Real-time events flow through src/lib/events/eventBus.ts and are accessible via src/hooks/useLiveDashboard.ts for dashboard visualization.
• The LIVE_DASHBOARD feature flag controls UI visibility without impacting core routing performance.
• All traffic monitoring occurs post-routing, ensuring zero latency impact on AI model selection.

Frequently Asked Questions

Does OmniRoute use live traffic latency to select AI models?

No. According to the source code in diegosouzapw/OmniRoute, the combo routing engine bases decisions on static metadata including provider capabilities, cost settings, and quota consumption. It does not ingest live traffic-volume or latency metrics when selecting models or determining request distribution.

How can I monitor real-time traffic in OmniRoute?

Enable the Traffic-Inspector feature and use the WebSocket endpoint at /api/tools/traffic-inspector/ws via the useLiveDashboard React hook. Alternatively, query the REST API endpoints or inspect the in-memory ring buffer using getProxyLogSnapshot from src/lib/proxyLogger.ts.

What is the Traffic-Inspector feature in OmniRoute?

The Traffic-Inspector is an observability toolset that captures every request and response, publishing them on a real-time event bus (src/lib/events/eventBus.ts) for dashboards, monitoring, and debugging. It includes WebSocket streaming, REST APIs, and an in-memory ring buffer for short-term traffic analysis.

Why doesn't OmniRoute factor real-time traffic into routing decisions?

The routing architecture prioritizes deterministic behavior and minimal latency. Incorporating real-time metrics would introduce processing overhead and potential latency spikes during the routing phase. Instead, OmniRoute separates concerns: static rules handle routing speed, while the Traffic-Inspector handles observability asynchronously.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}