OmniRoute Context-Relay Strategy: How It Preserves Session Continuity During Account Rotation

OmniRoute's context-relay strategy works in three stages: it monitors quota usage to detect when an account nears exhaustion, generates a compact handoff summary of the conversation history, and injects that summary into the first request of the new account—allowing users to experience seamless continuity even as the backend switches providers.

The context-relay routing strategy solves a critical problem in multi-account LLM deployments: maintaining conversation state when rotating between provider accounts due to quota limits. According to the OmniRoute source code, this mechanism captures chat context just before an account is exhausted and restores it on the next available account, effectively creating an invisible handoff that users never perceive.

How the Context-Relay Strategy Works

The implementation follows a deterministic three-stage pipeline that triggers automatically when a combo reaches its configured usage threshold.

Stage 1: Quota Monitoring and Threshold Detection

Each combo in OmniRoute continuously tracks the quota usage of its active provider account. When usage reaches the handoff threshold (defaulting to 85% of the quota), the context-relay strategy activates.

This threshold is defined as a configurable fraction in src/shared/constants/routingStrategies.ts (lines 112‑114). The combo evaluates this check before every request to determine if the active account is nearing exhaustion.

Stage 2: Handoff Summary Generation

Just before the active account would be exhausted, OmniRoute generates a compact handoff summary that captures the recent chat history. The system uses a configurable summary model—usually the same model configured for the combo, though this can be overridden via settings—to compress up to handoffMaxMessages (default 30) of conversation context.

The generation logic lives in src/lib/db/contextHandoffs.ts inside the maybeGenerateHandoff function. This function creates a structured <context_handoff> system message and persists it via upsertHandoff (see the insertion logic around line 250). The summary is stored in the Context‑Handoff DB, keyed by session ID and combo name, ensuring it remains available regardless of which account serves the next request.

Stage 3: Context Injection on Account Switch

When the next account in the combo becomes active, the stored handoff is automatically retrieved and injected as the first system message of the request payload. This gives the new provider the full conversational context of the previous exchange.

Injection is performed by injectUniversalHandoffBody in src/open-sse/services/combo.ts (around line 340). This function prepends the <context_handoff> message to the request body before forwarding it to the new provider, ensuring the user experiences a continuous session despite the backend switch.

Configurable Parameters for Context-Relay

The strategy exposes four key settings, available both in the UI and via the API:

Setting	Description	Default
handoffThreshold	Fraction of quota usage (0.0‑1.0) that triggers the handoff.	0.85
handoffMaxMessages	Maximum number of recent messages to compress into the handoff summary.	30
handoffSummaryModel	Optional model override used only for generating the summary.	Uses active combo model
handoffProviders	Array of provider IDs allowed to generate handoffs (e.g., ["codex"]). Empty array disables the feature.	["codex"]

When handoffProviders is empty, the strategy is effectively disabled. This behavior is verified by the unit test suite in tests/unit/combo-context-relay.test.ts (lines 340‑385), which asserts that no handoff is persisted when the provider list is empty.

Implementation Details and Code Examples

Enable Context-Relay via API

Configure a combo to use the context-relay strategy with custom thresholds:

POST /api/settings/combo-defaults
Content-Type: application/json

{
  "comboDefaults": {
    "routingStrategy": "contextRelay",
    "handoffThreshold": 0.9,
    "handoffMaxMessages": 20,
    "handoffProviders": ["codex"]
  }
}

Source: API handler in src/app/api/settings/combo-defaults/route.ts (see body handling around line 48).

Internal Handoff Injection

The server internally uses this pattern when switching accounts:

import { injectUniversalHandoffBody } from '@/open-sse/services/combo';

// `body` is the original chat request sent to the new account
const newBody = injectUniversalHandoffBody(body, handoffMessage);

Source: injectUniversalHandoffBody in src/open-sse/services/combo.ts (lines 340‑350).

Debug Stored Handoffs

To inspect what context was preserved during development:

import { getHandoff } from '@/lib/db/contextHandoffs';

const handoff = await getHandoff(sessionId, comboName);
console.log(handoff?.summary);   // prints the generated handoff text

Source: getHandoff in src/lib/db/contextHandoffs.ts (lines 60‑70).

Summary

• Context-relay is a priority-style routing strategy that preserves conversation continuity across account rotations by passing a compressed history summary between providers.
• The strategy triggers at a configurable quota threshold (default 85%), generates a summary via maybeGenerateHandoff, and stores it in the Context‑Handoff DB.
• The injectUniversalHandoffBody function in src/open-sse/services/combo.ts automatically injects this summary into requests sent to new accounts.
• Configuration is controlled through handoffThreshold, handoffMaxMessages, handoffSummaryModel, and handoffProviders settings.
• The mechanism is currently optimized for Codex-based rotations but supports any provider listed in handoffProviders.

Frequently Asked Questions

What triggers a context handoff in OmniRoute?

A context handoff triggers when the active provider account in a combo reaches the handoff threshold—by default, 85% of its allocated quota. The quota monitoring occurs in src/open-sse/services/combo.ts, which evaluates usage before each request and initiates the handoff process when the threshold is exceeded.

Which providers support the context-relay strategy?

Any provider included in the handoffProviders configuration array can participate in context-relay. By default, OmniRoute sets this to ["codex"], but you can extend it to include other provider IDs. An empty array disables the feature entirely, causing the combo to switch accounts without preserving conversation context.

How is the handoff summary generated?

The summary is generated by calling maybeGenerateHandoff in src/lib/db/contextHandoffs.ts. It uses either the combo's active model or a specific override defined in handoffSummaryModel to compress the last N messages (controlled by handoffMaxMessages) into a structured <context_handoff> system message. This message is then upserted into the database via upsertHandoff.

Can I disable context-relay for specific combos?

Yes. Set handoffProviders to an empty array ([]) in the combo configuration. This prevents the system from generating or injecting handoff summaries, effectively disabling the feature while maintaining standard account rotation behavior. The unit tests in tests/unit/combo-context-relay.test.ts validate this behavior, confirming that no handoff records are created when the provider list is empty.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}