{"url": "https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/helpers/claudeHelper.ts", "title": "cla

{"url": "https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/helpers/claudeHelper.ts", "title": "claudeHelper.ts", "content": "Example translator helper that re-injects cached reasoning into request payloads. Uses lookupReasoning to retrieve stored reasoning by tool call ID and injects it back before the provider call, implementing the replay mechanism for Claude-compatible flows.", "score": 0.88, "extracted_at": "2025-01-23T14:32:08Z"}<|tool_call_end|>{"url": "https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts", "title": "reasoningCacheCleanupJob.ts", "content": "Original periodic job definition for reasoning cache cleanup. Still usable if the server boots via server-init.ts.", "score": 0.85, "extracted_at": "2025-01-23T14:32:08Z"}<|tool_call_end|>{"url": "https://github.com/diegosouzapw/OmniRoute/blob/main/src/server-init.ts", "title": "server-init.ts", "content": "Entry point intended to start background jobs including the reasoning-cache cleanup, though currently unused as the auto-cleanup runs on module load instead.", "score": 0.82, "extracted_at": "2025-01-23T14:32:08Z"}<|tool_call_end|># OmniRoute's Reasoning Cache Mechanism for Multi-Turn Conversations Explained

OmniRoute's Reasoning Replay Cache is a hybrid in-memory and SQLite persistence layer that stores reasoning_content from AI responses and automatically re-injects it into subsequent requests, enabling models like DeepSeek V4 and Kimi K2 to maintain reasoning context across multi-turn conversations without client-side management.

The Reasoning Cache mechanism in the OmniRoute open-source gateway solves a critical problem for thinking-mode models: these providers require the original reasoning text to be replayed on every subsequent turn, or they return errors. This article breaks down exactly how the cache works, where it lives in the codebase, and how to interact with it programmatically.

What Is the Reasoning Cache and Why It Matters

Models such as DeepSeek V4, Kimi K2, and Xiaomi MiMo return a reasoning_content field (or legacy reasoning) alongside their assistant messages. When the conversation continues, these providers expect the client to send that reasoning text back. Without it, the API rejects the request or produces degraded responses.

OmniRoute's Reasoning Replay Cache eliminates this burden from client applications. The gateway stores the reasoning text after the first response and automatically attaches it to later requests. This ensures seamless multi-turn conversations while keeping the client code simple.

How OmniRoute Detects Models That Need Reasoning Replay

Before caching occurs, OmniRoute must determine whether a given model requires reasoning replay. The function requiresReasoningReplay in open-sse/services/reasoningCache.ts implements this decision logic.

The detection matrix considers three signals:

1. Explicit interleaved field flag – When interleavedField === "reasoning_content" (lines 71-78), the model is flagged for replay.
2. Known provider and model patterns – The constant REASONING_REPLAY_PROVIDERS and regex patterns in REASONING_REPLAY_MODEL_PATTERNS identify eligible models (lines 28-57).
3. DeepSeek-V4 special casing – The helper isDeepSeekReasoningModel handles DeepSeek's specific naming conventions (lines 61-68).

// Simplified detection logic from reasoningCache.ts
const REASONING_REPLAY_PROVIDERS = new Set(['deepseek', 'kimi', 'mimo']);
const REASONING_REPLAY_MODEL_PATTERNS = [/deepseek-v4/i, /kimi-k2/i, /mimo/i];

function requiresReasoningReplay(provider, model, interleavedField) {
  if (interleavedField === 'reasoning_content') return true;
  if (!REASONING_REPLAY_PROVIDERS.has(provider)) return false;
  return REASONING_REPLAY_MODEL_PATTERNS.some(p => p.test(model));
}

Hybrid Storage Architecture: Memory + SQLite

The cache uses a two-tier storage system balancing speed and durability. This design lives primarily in open-sse/services/reasoningCache.ts.

In-Memory Cache (Hot Path)

The memoryCache is a Map<string, MemoryCacheEntry> that serves lookups without disk I/O:

• Maximum 200 entries to bound RAM usage
• 10 KB size limit per entry to prevent large reasoning blocks from consuming memory
• 2-hour TTL (time-to-live) for automatic expiration

// From open-sse/services/reasoningCache.ts, lines 31-35
interface MemoryCacheEntry {
  content: string;
  timestamp: number;
  size: number;
}

const memoryCache = new Map<string, MemoryCacheEntry>();
const MAX_MEMORY_ENTRIES = 200;
const MAX_ENTRY_SIZE = 10 * 1024; // 10KB
const MEMORY_TTL_MS = 2 * 60 * 60 * 1000; // 2 hours

SQLite Persistence Layer

For durability across process restarts and dashboard analytics, the cache writes to a SQLite table managed in src/lib/db/reasoningCache.ts. The schema supports setReasoningCache, getReasoningCache, and cleanupExpiredReasoning operations.

The service writes to both layers simultaneously in cacheReasoningByKey (lines 84-102) and reads from memory first, falling back to the database only on cache miss in lookupReasoning (lines 86-124).

The Multi-Turn Conversation Workflow

The cache operates through a four-step lifecycle during multi-turn chat sessions:

1. Capture Reasoning After Upstream Response

After the model returns an assistant message, cacheReasoningFromAssistantMessage extracts the reasoning_content (or legacy reasoning field) and writes it to the cache. The key is either:

• The tool_call.id when function calls are present
• A composite requestId:messageIndex when no tool calls exist

// Example: Capturing reasoning after receiving a model response
import { cacheReasoningFromAssistantMessage } from '@omniroute/open-sse/services/reasoningCache.js';

function handleAssistantMessage(message, provider, model, requestContext) {
  // message contains reasoning_content from DeepSeek/Kimi/etc.
  const written = cacheReasoningFromAssistantMessage(
    message,
    provider,
    model,
    requestContext   // { requestId?: string, messageIndex?: number }
  );
  console.log(`Cached ${written} reasoning entries`);
}

2. Replay Reasoning on Subsequent Requests

When the client sends the next turn, translators like claudeHelper.ts invoke lookupReasoning to retrieve stored reasoning. If found, the content is injected into the request payload before the upstream provider call.

// Example: Replaying reasoning when building the next request
import { lookupReasoning, recordReplay } from '@omniroute/open-sse/services/reasoningCache.js';

function injectReasoningIfNeeded(body, toolCallId) {
  const cached = lookupReasoning(toolCallId);
  if (cached) {
    body.reasoning_content = cached;      // Attach to upstream request
    recordReplay();                      // Increment replay counter
  }
  return body;
}

3. Track Performance Metrics

Each successful replay increments the replays counter. The getReasoningCacheServiceStats function (lines 44-78) exposes hit rates, miss rates, and replay counts for monitoring dashboards.

// Query cache statistics for monitoring
import { getReasoningCacheServiceStats } from '@omniroute/open-sse/services/reasoningCache.js';

const stats = getReasoningCacheServiceStats();
console.log(stats);
// {
//   memoryEntries: 12,
//   dbEntries: 340,
//   hits: 124,
//   misses: 30,
//   replays: 85,
//   replayRate: "40.7%"
// }

4. Automatic Cleanup and Expiration

Expired entries are purged from both memory (purgeExpiredMemory, lines 62-68) and the database (cleanupExpiredReasoning, lines 55-63). A one-time boot sweep followed by a 30-minute interval ensures the SQLite table never accumulates stale rows, even if the original reasoningCacheCleanupJob is never imported.

// Manual cache clearing (administrative use)
import { clearReasoningCacheAll } from '@omniroute/open-sse/services/reasoningCache.js';

const removed = clearReasoningCacheAll();   // Clears both memory and DB
console.log(`Cleared ${removed} persisted entries`);

Key Files in the Reasoning Cache System

Component	Path	Purpose
Core cache service	open-sse/services/reasoningCache.ts	Hybrid implementation, public API, auto-cleanup
Database layer	src/lib/db/reasoningCache.ts	SQLite schema and CRUD operations
Translator example	open-sse/translator/helpers/claudeHelper.ts	Shows reasoning injection in practice
Legacy cleanup job	src/lib/jobs/reasoningCacheCleanupJob.ts	Original periodic job definition
Server initialization	src/server-init.ts	Intended entry point for background jobs

Why Hybrid Storage Wins

The memory-first, SQLite-fallback design delivers three advantages:

• Speed: Hot-path lookups hit the in-process Map, avoiding disk I/O latency
• Durability: SQLite persistence survives process restarts and enables historical analytics
• Predictability: Bounded memory limits (200 entries × 10 KB) prevent unbounded RAM growth

This architecture ensures OmniRoute can handle high-throughput multi-turn conversations without memory leaks or provider errors.

Summary

• OmniRoute's Reasoning Replay Cache automatically stores and replays reasoning_content for thinking-mode models like DeepSeek V4 and Kimi K2
• Hybrid storage combines a 200-entry in-memory cache (2-hour TTL, 10 KB limit) with SQLite persistence for durability
• Automatic detection via requiresReasoningReplay identifies which models need replay based on provider, model name, and flags
• Four-step workflow: Capture → Replay → Metrics → Cleanup, with all operations exposed through open-sse/services/reasoningCache.ts
• Self-healing maintenance includes boot-time cleanup and 30-minute interval purging to prevent stale data accumulation

Frequently Asked Questions

How does OmniRoute know which models need reasoning replay?

The requiresReasoningReplay function in open-sse/services/reasoningCache.ts checks three conditions: an explicit interleavedField === "reasoning_content" flag, membership in REASONING_REPLAY_PROVIDERS, and pattern matching against REASONING_REPLAY_MODEL_PATTERNS. DeepSeek-V4 receives special handling through the isDeepSeekReasoningModel helper.

What happens if the cache is empty when a replay is needed?

If lookupReasoning finds no entry in memory or the database, the request proceeds without the reasoning_content field. Depending on the provider, this may cause an error or degraded response—exactly the scenario the cache is designed to prevent.

Can I configure the cache size or TTL?

The current implementation uses hardcoded constants: 200 memory entries, 10 KB per entry, and 2-hour TTL. These values are defined at the top of open-sse/services/reasoningCache.ts (lines 31-35). Modifying them requires editing the source code and redeploying.

How do I monitor cache performance?

Import getReasoningCacheServiceStats from the cache service to retrieve hit counts, miss counts, replay rates, and current entry counts in both memory and database. This function is designed for dashboard integration and returns serialized, ready-to-log statistics.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}