# {"url": "https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/helpers/claudeHelper.ts", "title": "cla

> Discover OmniRoute's innovative reasoning cache mechanism. This helper injects stored reasoning into Claude-compatible flows, enabling efficient multi-turn conversations by replaying past logic.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-07-03

---

{"url": "https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/helpers/claudeHelper.ts", "title": "claudeHelper.ts", "content": "Example translator helper that re-injects cached reasoning into request payloads. Uses `lookupReasoning` to retrieve stored reasoning by tool call ID and injects it back before the provider call, implementing the replay mechanism for Claude-compatible flows.", "score": 0.88, "extracted_at": "2025-01-23T14:32:08Z"}<|tool_call_end|>{"url": "https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts", "title": "reasoningCacheCleanupJob.ts", "content": "Original periodic job definition for reasoning cache cleanup. Still usable if the server boots via [`server-init.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/server-init.ts).", "score": 0.85, "extracted_at": "2025-01-23T14:32:08Z"}<|tool_call_end|>{"url": "https://github.com/diegosouzapw/OmniRoute/blob/main/src/server-init.ts", "title": "server-init.ts", "content": "Entry point intended to start background jobs including the reasoning-cache cleanup, though currently unused as the auto-cleanup runs on module load instead.", "score": 0.82, "extracted_at": "2025-01-23T14:32:08Z"}<|tool_call_end|># OmniRoute's Reasoning Cache Mechanism for Multi-Turn Conversations Explained

**OmniRoute's Reasoning Replay Cache is a hybrid in-memory and SQLite persistence layer that stores `reasoning_content` from AI responses and automatically re-injects it into subsequent requests, enabling models like DeepSeek V4 and Kimi K2 to maintain reasoning context across multi-turn conversations without client-side management.**

The **Reasoning Cache** mechanism in the [OmniRoute](https://github.com/diegosouzapw/OmniRoute) open-source gateway solves a critical problem for *thinking-mode* models: these providers require the original reasoning text to be replayed on every subsequent turn, or they return errors. This article breaks down exactly how the cache works, where it lives in the codebase, and how to interact with it programmatically.

## What Is the Reasoning Cache and Why It Matters

Models such as **DeepSeek V4**, **Kimi K2**, and **Xiaomi MiMo** return a `reasoning_content` field (or legacy `reasoning`) alongside their assistant messages. When the conversation continues, these providers expect the client to send that reasoning text back. Without it, the API rejects the request or produces degraded responses.

OmniRoute's **Reasoning Replay Cache** eliminates this burden from client applications. The gateway stores the reasoning text after the first response and automatically attaches it to later requests. This ensures seamless multi-turn conversations while keeping the client code simple.

## How OmniRoute Detects Models That Need Reasoning Replay

Before caching occurs, OmniRoute must determine whether a given model requires reasoning replay. The function `requiresReasoningReplay` in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) implements this decision logic.

The detection matrix considers three signals:

1. **Explicit interleaved field flag** – When `interleavedField === "reasoning_content"` (lines 71-78), the model is flagged for replay.
2. **Known provider and model patterns** – The constant `REASONING_REPLAY_PROVIDERS` and regex patterns in `REASONING_REPLAY_MODEL_PATTERNS` identify eligible models (lines 28-57).
3. **DeepSeek-V4 special casing** – The helper `isDeepSeekReasoningModel` handles DeepSeek's specific naming conventions (lines 61-68).

```typescript
// Simplified detection logic from reasoningCache.ts
const REASONING_REPLAY_PROVIDERS = new Set(['deepseek', 'kimi', 'mimo']);
const REASONING_REPLAY_MODEL_PATTERNS = [/deepseek-v4/i, /kimi-k2/i, /mimo/i];

function requiresReasoningReplay(provider, model, interleavedField) {
  if (interleavedField === 'reasoning_content') return true;
  if (!REASONING_REPLAY_PROVIDERS.has(provider)) return false;
  return REASONING_REPLAY_MODEL_PATTERNS.some(p => p.test(model));
}

```

## Hybrid Storage Architecture: Memory + SQLite

The cache uses a **two-tier storage system** balancing speed and durability. This design lives primarily in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts).

### In-Memory Cache (Hot Path)

The `memoryCache` is a `Map<string, MemoryCacheEntry>` that serves lookups without disk I/O:

- **Maximum 200 entries** to bound RAM usage
- **10 KB size limit per entry** to prevent large reasoning blocks from consuming memory
- **2-hour TTL** (time-to-live) for automatic expiration

```typescript
// From open-sse/services/reasoningCache.ts, lines 31-35
interface MemoryCacheEntry {
  content: string;
  timestamp: number;
  size: number;
}

const memoryCache = new Map<string, MemoryCacheEntry>();
const MAX_MEMORY_ENTRIES = 200;
const MAX_ENTRY_SIZE = 10 * 1024; // 10KB
const MEMORY_TTL_MS = 2 * 60 * 60 * 1000; // 2 hours

```

### SQLite Persistence Layer

For durability across process restarts and dashboard analytics, the cache writes to a SQLite table managed in [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts). The schema supports `setReasoningCache`, `getReasoningCache`, and `cleanupExpiredReasoning` operations.

The service writes to **both layers simultaneously** in `cacheReasoningByKey` (lines 84-102) and reads from memory first, falling back to the database only on cache miss in `lookupReasoning` (lines 86-124).

## The Multi-Turn Conversation Workflow

The cache operates through a four-step lifecycle during multi-turn chat sessions:

### 1. Capture Reasoning After Upstream Response

After the model returns an assistant message, `cacheReasoningFromAssistantMessage` extracts the `reasoning_content` (or legacy `reasoning` field) and writes it to the cache. The key is either:
- The `tool_call.id` when function calls are present
- A composite `requestId:messageIndex` when no tool calls exist

```typescript
// Example: Capturing reasoning after receiving a model response
import { cacheReasoningFromAssistantMessage } from '@omniroute/open-sse/services/reasoningCache.js';

function handleAssistantMessage(message, provider, model, requestContext) {
  // message contains reasoning_content from DeepSeek/Kimi/etc.
  const written = cacheReasoningFromAssistantMessage(
    message,
    provider,
    model,
    requestContext   // { requestId?: string, messageIndex?: number }
  );
  console.log(`Cached ${written} reasoning entries`);
}

```

### 2. Replay Reasoning on Subsequent Requests

When the client sends the next turn, translators like [`claudeHelper.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/claudeHelper.ts) invoke `lookupReasoning` to retrieve stored reasoning. If found, the content is injected into the request payload before the upstream provider call.

```typescript
// Example: Replaying reasoning when building the next request
import { lookupReasoning, recordReplay } from '@omniroute/open-sse/services/reasoningCache.js';

function injectReasoningIfNeeded(body, toolCallId) {
  const cached = lookupReasoning(toolCallId);
  if (cached) {
    body.reasoning_content = cached;      // Attach to upstream request
    recordReplay();                      // Increment replay counter
  }
  return body;
}

```

### 3. Track Performance Metrics

Each successful replay increments the `replays` counter. The `getReasoningCacheServiceStats` function (lines 44-78) exposes hit rates, miss rates, and replay counts for monitoring dashboards.

```typescript
// Query cache statistics for monitoring
import { getReasoningCacheServiceStats } from '@omniroute/open-sse/services/reasoningCache.js';

const stats = getReasoningCacheServiceStats();
console.log(stats);
// {
//   memoryEntries: 12,
//   dbEntries: 340,
//   hits: 124,
//   misses: 30,
//   replays: 85,
//   replayRate: "40.7%"
// }

```

### 4. Automatic Cleanup and Expiration

Expired entries are purged from both memory (`purgeExpiredMemory`, lines 62-68) and the database (`cleanupExpiredReasoning`, lines 55-63). A one-time boot sweep followed by a 30-minute interval ensures the SQLite table never accumulates stale rows, even if the original `reasoningCacheCleanupJob` is never imported.

```typescript
// Manual cache clearing (administrative use)
import { clearReasoningCacheAll } from '@omniroute/open-sse/services/reasoningCache.js';

const removed = clearReasoningCacheAll();   // Clears both memory and DB
console.log(`Cleared ${removed} persisted entries`);

```

## Key Files in the Reasoning Cache System

| Component | Path | Purpose |
|-----------|------|---------|
| Core cache service | [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) | Hybrid implementation, public API, auto-cleanup |
| Database layer | [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts) | SQLite schema and CRUD operations |
| Translator example | [`open-sse/translator/helpers/claudeHelper.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/translator/helpers/claudeHelper.ts) | Shows reasoning injection in practice |
| Legacy cleanup job | [`src/lib/jobs/reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts) | Original periodic job definition |
| Server initialization | [`src/server-init.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/server-init.ts) | Intended entry point for background jobs |

## Why Hybrid Storage Wins

The **memory-first, SQLite-fallback** design delivers three advantages:

- **Speed**: Hot-path lookups hit the in-process `Map`, avoiding disk I/O latency
- **Durability**: SQLite persistence survives process restarts and enables historical analytics
- **Predictability**: Bounded memory limits (200 entries × 10 KB) prevent unbounded RAM growth

This architecture ensures OmniRoute can handle high-throughput multi-turn conversations without memory leaks or provider errors.

## Summary

- **OmniRoute's Reasoning Replay Cache** automatically stores and replays `reasoning_content` for thinking-mode models like DeepSeek V4 and Kimi K2
- **Hybrid storage** combines a 200-entry in-memory cache (2-hour TTL, 10 KB limit) with SQLite persistence for durability
- **Automatic detection** via `requiresReasoningReplay` identifies which models need replay based on provider, model name, and flags
- **Four-step workflow**: Capture → Replay → Metrics → Cleanup, with all operations exposed through [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts)
- **Self-healing maintenance** includes boot-time cleanup and 30-minute interval purging to prevent stale data accumulation

## Frequently Asked Questions

### How does OmniRoute know which models need reasoning replay?

The `requiresReasoningReplay` function in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) checks three conditions: an explicit `interleavedField === "reasoning_content"` flag, membership in `REASONING_REPLAY_PROVIDERS`, and pattern matching against `REASONING_REPLAY_MODEL_PATTERNS`. DeepSeek-V4 receives special handling through the `isDeepSeekReasoningModel` helper.

### What happens if the cache is empty when a replay is needed?

If `lookupReasoning` finds no entry in memory or the database, the request proceeds without the `reasoning_content` field. Depending on the provider, this may cause an error or degraded response—exactly the scenario the cache is designed to prevent.

### Can I configure the cache size or TTL?

The current implementation uses hardcoded constants: 200 memory entries, 10 KB per entry, and 2-hour TTL. These values are defined at the top of [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) (lines 31-35). Modifying them requires editing the source code and redeploying.

### How do I monitor cache performance?

Import `getReasoningCacheServiceStats` from the cache service to retrieve hit counts, miss counts, replay rates, and current entry counts in both memory and database. This function is designed for dashboard integration and returns serialized, ready-to-log statistics.