How OmniRoute's Context-Relay Routing Strategy Maintains Conversation Context

OmniRoute's context-relay routing strategy preserves conversation state by storing a "handoff" snapshot of the previous provider's context in a database, then injecting that snapshot into subsequent requests when the combo switches to a different account.

OmniRoute is an open-source routing layer that enables requests to traverse multiple AI provider accounts within a single combo. The context-relay routing strategy ensures that when these combos switch between provider accounts, the conversational context remains intact. This mechanism prevents dialogue fragmentation by maintaining state across disparate provider connections.

How Context-Relay Works: The Handoff Lifecycle

Strategy Activation

When a request arrives, the routing engine examines the combo's strategy field. If the value equals "context-relay", the request handler activates specialized logic in src/sse/handlers/chat.ts at lines 1298-1310. This triggers the context preservation workflow before the request is forwarded upstream.

Session and Combo Identification

Every context-relay request requires two identifiers: a runtimeOptions.sessionId (typically a UUID) and a comboName. These parameters identify the logical conversation thread and the specific combo configuration being used. Without these identifiers, the system cannot correlate requests across account switches.

Retrieving Stored Context

Before sending the request upstream, the handler invokes getHandoff(runtimeOptions.sessionId, comboName) from src/lib/db/contextHandoffs.ts (lines 21-34). This function queries the context_handoffs table for a non-expired snapshot containing the previous provider's context, including summaries, key decisions, active entities, and the model used. The handoff only returns if expires_at > now.

Detecting Account Switches

The system compares the retrieved handoff's fromAccount field against the current connection's connectionId. If they differ, OmniRoute recognizes that the conversation has moved to a new provider account and requires context injection to maintain continuity.

Injecting Handoff Data

Upon detecting a switch, the handler calls injectHandoffIntoBody(requestBody, handoff) within src/sse/handlers/chat.ts (lines 1304-1317). This merges the handoff payload into the upstream request body, preserving prior system messages, tool results, or user-provided summaries. The request is then marked with _omnirouteSkipContextRelay to prevent recursive injection. A log entry records the transfer for observability (lines 1310-1316).

Persisting Updated Context

After receiving the provider's response, the combo router calls upsertHandoff(payload) (defined in src/lib/db/contextHandoffs.ts, lines 80-93) to persist the latest context snapshot. This overwrites any previous entry for the same session and combo combination, ensuring the next hop receives current state information.

Database Maintenance and Cleanup

The context_handoffs table requires periodic maintenance to prevent bloat. The cleanupExpiredHandoffs() function (lines 45-56 in src/lib/db/contextHandoffs.ts) removes stale entries where expires_at has passed, keeping the database performant and storage costs predictable.

Configuration and Usage Examples

Enabling context-relay requires specific combo configuration and runtime parameters.

Defining a Context-Relay Combo

Configure your combo with the strategy field set to "context-relay":

{
  "name": "my-relay-combo",
  "strategy": "context-relay",
  "targets": [
    { "provider": "openai", "model": "gpt-4o", "connectionId": "acct-1" },
    { "provider": "anthropic", "model": "claude-3-sonnet", "connectionId": "acct-2" }
  ]
}

Triggering Context Preservation

Include the session identifier in your API request to enable handoff detection:

await fetch("https://api.omniroute.dev/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "my-relay-combo",
    messages: [{ role: "user", content: "Continue the story." }],
    _omnirouteSessionId: "a1b2c3d4-e5f6-7890-abcd-1234567890ab"
  })
});

Debugging Handoff State

Inspect stored handoffs for troubleshooting:

import { getHandoff } from "./src/lib/db/contextHandoffs";

const handoff = getHandoff("a1b2c3d4-e5f6-7890-abcd-1234567890ab", "my-relay-combo");
console.log(handoff);

Summary

• OmniRoute's context-relay strategy uses a handoff mechanism to preserve conversation state across provider account switches.
• The system stores context snapshots in the context_handoffs table via upsertHandoff() and retrieves them using getHandoff().
• Account switches are detected by comparing fromAccount in the stored handoff against the current connectionId.
• Context is injected into requests through injectHandoffIntoBody() in src/sse/handlers/chat.ts.
• Expired handoffs are automatically purged by cleanupExpiredHandoffs() to maintain database health.

Frequently Asked Questions

What happens if no sessionId is provided?

Without a runtimeOptions.sessionId, the context-relay strategy cannot correlate requests across account switches. The system will treat each request as a new conversation, and no handoff injection will occur.

How long are handoff contexts retained?

Handoffs persist until their expires_at timestamp is reached. The cleanupExpiredHandoffs() function periodically removes stale entries, though the exact TTL depends on your OmniRoute configuration.

Can context-relay work with more than two provider accounts?

Yes. The strategy supports combos with multiple targets. Each account switch triggers a new handoff injection, allowing conversations to traverse unlimited provider hops while maintaining continuity.

Does context-relay affect response latency?

Minimal overhead occurs during handoff retrieval and injection. The database lookup in getHandoff() is indexed by session and combo, and the payload merge happens in-memory before the upstream request.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}