How the Context Relay System Maintains Session Continuity During Account Rotation in OmniRoute

The Context Relay system preserves conversation flow during account rotation by generating handoff summaries at the combo layer when quota thresholds are met, storing them in SQLite, and injecting them into subsequent requests at the chat layer after provider resolution.

The OmniRoute repository implements a sophisticated Context Relay mechanism to solve the problem of session fragmentation when AI provider accounts rotate mid-conversation. By splitting responsibility between the routing decision layer and the request handling layer, the system ensures that new accounts receive compressed context summaries of prior interactions without duplicating requests or losing conversational state.

The Combo Layer: Detecting Rotation and Queuing Handoffs

In src/shared/constants/routingStrategies.ts, the strategy is identified by the label "contextRelay". The combo layer in open-sse/services/combo.ts monitors real-time quota consumption against the configurable handoffThreshold parameter.

When usage crosses this threshold, the system triggers a background summary generation using the handoffModel (defaulting to gpt-4o-mini). This process compresses the conversation history into a concise handoff summary and persists it to the database via src/lib/db/context_handoffs.ts.

The Chat Layer: Injecting Context After Account Resolution

Only open-sse/handlers/chat.ts knows whether an actual account change occurred during routing resolution. After the new provider account is selected, this handler queries the context_handoffs SQLite table to retrieve the latest entry for the current session_id. It then prepends the stored summary to the message array using a system role prefix, ensuring the new account receives the full conversational context before processing the user request.

Database Schema for Session Continuity

The context_handoffs table defined in src/lib/db/context_handoffs.ts maintains the following schema:

• combo_id: Identifier for the combo configuration that generated the handoff
• session_id: Unique session identifier linking the handoff to the user conversation
• summary: The compressed conversation text generated by the handoff model
• created_at: Timestamp of generation used for ordering and cleanup

This storage strategy decouples summary generation from consumption, preventing race conditions when multiple requests occur during account rotation.

Configuration and Code Examples

Enable Context Relay in your combo configuration:

// Configuration example for combo setup
import { ComboConfig } from '@/open-sse/config/comboConfig';

export const productionCombo: ComboConfig = {
  strategy: 'contextRelay',               // Activates handoff logic
  handoffThreshold: 0.75,                // Trigger at 75% quota usage
  handoffModel: 'gpt-4o-mini',           // Model for summary generation
  handoffProviders: ['openai', 'anthropic'] // Whitelist eligible providers
};

The combo layer generates and stores handoffs:

// Conceptual implementation flow from open-sse/services/combo.ts
if (currentUsage >= combo.handoffThreshold) {
  const summary = await generateSummary({
    model: combo.handoffModel,
    messages: conversationHistory
  });
  
  await db.insertContextHandoff({
    combo_id: combo.id,
    session_id: sessionId,
    summary: summary.content,
    created_at: new Date()
  });
}

The chat handler injects stored context:

// Implementation pattern from open-sse/handlers/chat.ts
import { getLatestHandoff } from '@/lib/db/context_handoffs';

const handoff = await getLatestHandoff(request.sessionId);
if (handoff) {
  request.body.messages.unshift({
    role: 'system',
    content: `[Context Handoff] ${handoff.summary}`
  });
}

Key Source Files

Component	File Path	Purpose
Strategy Definition	src/shared/constants/routingStrategies.ts	Defines the "contextRelay" strategy identifier
Handoff Generation	open-sse/services/combo.ts	Monitors thresholds and queues summary creation
Context Injection	open-sse/handlers/chat.ts	Retrieves and injects handoffs after account resolution
Data Persistence	src/lib/db/context_handoffs.ts	SQLite schema and queries for handoff storage
Documentation	docs/i18n/zh-CN/docs/features/context-relay.md	Multi-language feature documentation

Summary

• The Context Relay system splits responsibility between the combo layer (generation) and chat layer (injection) to maintain session continuity.
• Threshold-based detection in open-sse/services/combo.ts triggers summary generation when quota usage crosses handoffThreshold.
• Summaries are stored in the context_handoffs SQLite table with session_id linkage for retrieval.
• The open-sse/handlers/chat.ts handler injects stored context only after confirming account rotation, preventing context loss.
• Configuration through strategy: 'contextRelay' supports custom models via handoffModel and provider whitelisting via handoffProviders.

Frequently Asked Questions

What triggers a context handoff in OmniRoute?

A context handoff triggers when the combo layer detects that quota usage has exceeded the handoffThreshold percentage (e.g., 0.75 for 75%). At this point, open-sse/services/combo.ts initiates a summary request to the configured handoffModel and stores the result in the context_handoffs table before the account actually rotates.

How does the Context Relay system prevent duplicate handoffs?

The architecture separates generation from injection. While combo.ts may generate a summary when thresholds are met, only chat.ts performs the actual injection after confirming account resolution. This ensures that handoffs are inserted exactly once per rotation event, even if multiple requests occur during the transition window.

Can I customize the model used for generating handoff summaries?

Yes. Configure the handoffModel property in your combo configuration (e.g., handoffModel: 'gpt-4o-mini'). The system passes this model identifier to the summary generation function in open-sse/services/combo.ts, allowing you to balance cost and quality based on your specific requirements.

Where is the handoff data stored between account rotations?

Handoff data persists in the context_handoffs SQLite table defined in src/lib/db/context_handoffs.ts. The table schema includes combo_id, session_id, summary, and created_at columns, enabling the open-sse/handlers/chat.ts handler to retrieve the most recent context for any given session ID during the next request cycle.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}