# Cache-Optimized vs Context-Optimized Routing in OmniRoute: A Complete Comparison

> Discover the difference between cache-optimized and context-optimized routing in OmniRoute. Learn how cache optimization maximizes prompt reuse while context optimization leverages larger context windows for better performance.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: comparison
- Published: 2026-08-05

---

**Cache-optimized routing pins requests to the same provider/account to maximize prompt-cache reuse, while context-optimized routing prioritizes models with the largest available context windows.**

OmniRoute, the open-source AI routing engine by `diegosouzapw/OmniRoute`, provides multiple routing strategies that determine how a **combo** selects its target provider and model. Two specialized strategies—**cache-optimized** and **context-optimized**—address distinct performance needs: cache locality versus context capacity. Both strategies are defined in [[`src/shared/constants/routingStrategies.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/routingStrategies.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/shared/constants/routingStrategies.ts#L95-L102) and implemented in the combo dispatcher via `applyStrategyOrdering`.

## What Is Cache-Optimized Routing?

Cache-optimized routing maximizes **prompt-cache reuse** by keeping consecutive requests on the same provider/account. This reduces latency when the same prompt prefix is used repeatedly.

### How Cache-Optimized Routing Works

The implementation in [[`open-sse/services/combo/promptCacheAffinity.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/promptCacheAffinity.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/promptCacheAffinity.ts) follows four steps:

1. **Resolve a cache affinity key** from the request body—either an explicit `prompt_cache_key` or a deterministic prefix hash—via `resolvePromptCacheAffinityKey` ([lines 94-115](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/promptCacheAffinity.ts#L94-L115))

2. **Expand the target list** with concrete active connections using `expandPromptCacheAffinityTargets` to enable affinity targeting at the account level ([lines 164-176](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/promptCacheAffinity.ts#L164-L176))

3. **Apply rendezvous hashing** via `applyPromptCacheAffinity`, scoring targets with the resolved key; the highest-scoring target becomes first in the ordered list ([lines 73-100](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/promptCacheAffinity.ts#L73-L100))

4. Return the ordered list to the combo executor

The ordering is **stable for a given cache key**—repeated calls with identical keys hit the same account. The combo logger emits: `Cache‑optimized ordering: … first` ([source](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/applyStrategyOrdering.ts#L204-L213)).

### When to Use Cache-Optimized Routing

- **Chatbots with consistent prefixes** that reuse the same system prompt
- **High-frequency similar requests** where provider-side prompt embeddings provide measurable latency reduction
- Workloads where **cache locality outweighs cost or context considerations**

Cache-optimized routing is **mutually exclusive** with most other ordering strategies. It runs after primary ordering (e.g., cost-optimized) and overrides the result based on cache affinity.

## What Is Context-Optimized Routing?

Context-optimized routing prioritizes the **largest available context window**, ensuring the model can process the maximum amount of conversation history or document content.

### How Context-Optimized Routing Works

The implementation centers on [[`open-sse/services/combo/comboStructure.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/comboStructure.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/comboStructure.ts):

1. Retrieve each target's **known context limit** via `getModelContextLimitForModelString`

2. **Sort all targets by context window size** in descending order using `sortTargetsByContextSize` ([lines 13-31](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/comboStructure.ts#L13-L31))

3. Return the reordered list with the largest-context model first

The logger emits: `Context‑optimized ordering: largest first …` ([source](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/applyStrategyOrdering.ts#L200-L203)).

### When to Use Context-Optimized Routing

- **Long-form document analysis** requiring maximum token capacity
- **Multi-turn conversations** with extensive history
- Scenarios where **context capacity matters more than cache locality or cost**

Context-optimized routing is a **stand-alone ordering strategy** that directly replaces any previous ordering. It does not combine with modifiers like cost-optimized because it completely reorders the target list.

## Key Differences: Cache-Optimized vs Context-Optimized

| Aspect | Cache-Optimized | Context-Optimized |
|--------|-----------------|-------------------|
| **Primary goal** | Maximize prompt-cache reuse on the same provider/account | Maximize available context window size |
| **Ordering mechanism** | Rendezvous hashing with stable affinity key | Descending sort by context window size |
| **Effect on target list** | Pins first target; preserves subsequent order | Completely reorders entire list |
| **Interaction with other strategies** | Runs after and overrides other orderings | Replaces all other ordering logic |
| **Best for** | Repeated similar prompts, cache-sensitive workloads | Long documents, multi-turn conversations |
| **Core source file** | [`open-sse/services/combo/promptCacheAffinity.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/promptCacheAffinity.ts) | [`open-sse/services/combo/comboStructure.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/comboStructure.ts) |

## Code Examples: Defining Each Strategy

```typescript
// Cache-optimized: keeps requests on same provider for cache warmth
const comboCacheOptimized = {
  name: "chat-cache-opt",
  strategy: "cache-optimized",
  targets: [
    { provider: "openrouter", modelStr: "anthropic/claude-3.5-sonnet" },
    { provider: "openrouter", modelStr: "openai/gpt-4o-mini" },
  ],
};

// Context-optimized: prioritizes largest context window
const comboContextOptimized = {
  name: "doc-review-context-opt",
  strategy: "context-optimized",
  targets: [
    { provider: "openrouter", modelStr: "google/gemini-2.5-flash" },  // 1,048,576 tokens
    { provider: "openrouter", modelStr: "openai/gpt-4o-mini" },       // 128,000 tokens
  ],
};

// Both are invoked through the same handler
await handleComboChat(buildRequest({ body: { model: "chat-cache-opt", ... } }));
await handleComboChat(buildRequest({ body: { model: "doc-review-context-opt", ... } }));

```

## Source Code Reference

Understanding the implementation requires familiarity with these key files in `diegosouzapw/OmniRoute`:

| File | Purpose |
|------|---------|
| [[`src/shared/constants/routingStrategies.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/shared/constants/routingStrategies.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/shared/constants/routingStrategies.ts) | Enum defining `cache-optimized` and `context-optimized` as valid strategy values |
| [[`open-sse/services/combo/applyStrategyOrdering.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/applyStrategyOrdering.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/applyStrategyOrdering.ts) | Central dispatch implementing both strategy branches |
| [[`open-sse/services/combo/promptCacheAffinity.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/promptCacheAffinity.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/promptCacheAffinity.ts) | Cache affinity resolution, expansion, and rendezvous hashing |
| [[`open-sse/services/combo/comboStructure.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/combo/comboStructure.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/comboStructure.ts) | `sortTargetsByContextSize` for context-based ordering |

## Summary

- **Cache-optimized routing** leverages `resolvePromptCacheAffinityKey`, `expandPromptCacheAffinityTargets`, and `applyPromptCacheAffinity` to pin requests to consistent provider accounts, reducing latency through prompt-cache reuse.

- **Context-optimized routing** uses `sortTargetsByContextSize` in [`comboStructure.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/comboStructure.ts) to reorder targets purely by context window capacity, ensuring maximum available memory for each request.

- The strategies are **fundamentally incompatible** with each other—cache-optimized modifies existing ordering while context-optimized replaces it entirely.

- Choose cache-optimized for **repetitive, cache-sensitive workloads** and context-optimized for **memory-intensive, long-context applications**.

## Frequently Asked Questions

### Can I combine cache-optimized and context-optimized routing in the same combo?

No. According to the OmniRoute source code, these strategies operate differently—cache-optimized runs after and overrides other orderings, while context-optimized completely replaces the target list order. The `applyStrategyOrdering` dispatcher processes only one strategy per combo request. You must select the strategy that better aligns with your performance priority.

### How does cache-optimized routing handle multiple active connections to the same provider?

The `expandPromptCacheAffinityTargets` function ([lines 164-176](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/promptCacheAffinity.ts#L164-L176)) expands the abstract target list into concrete active connections, enabling affinity at the account level. Rendezvous hashing then deterministically selects one specific connection based on the resolved affinity key, ensuring stable routing even with multiple endpoints.

### Does context-optimized routing consider cost or latency?

No. The `sortTargetsByContextSize` implementation ([lines 13-31](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/combo/comboStructure.ts#L13-L31)) sorts purely by context window size in descending order. It does not factor in pricing, latency, or other performance metrics. For cost-conscious context-heavy workloads, you may need to manually order targets or use a different strategy.

### What happens if the cache-optimized target becomes unavailable?

The cache-optimized strategy produces an **ordered list**, not a single target. If the affinity-pinned first target fails, the combo executor falls through to subsequent targets in the ordered list. However, fallback targets will not share the same prompt-cache state, potentially increasing latency for that specific request.