# How OmniRoute's Reasoning Replay Re‑Injects Cached Reasoning for Multi‑Turn Conversations

> Discover how OmniRoute's Reasoning Replay Cache re-injects cached reasoning for multi-turn conversations, satisfying model contracts like DeepSeek V4 and Kimi-K2.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-03

---

**OmniRoute's Reasoning Replay Cache automatically stores a model's `reasoning_content` when tool‑calling responses are received and re‑injects that content on later turns where the client omits it, satisfying the strict "reasoning must be passed back" contract required by DeepSeek V4, Kimi‑K2, and other thinking‑mode models.**

OmniRoute's **reasoning replay** mechanism solves a critical compatibility problem in multi‑turn conversations with modern LLMs that expose their internal reasoning chains. When models like DeepSeek V4 return `reasoning_content` alongside tool calls, subsequent API calls must include that reasoning or face 400 errors. OmniRoute's hybrid in‑memory and SQLite cache makes this requirement transparent to clients by automatically preserving and replaying reasoning across conversation turns.

## How Reasoning Replay Detection Works

OmniRoute identifies which provider and model combinations require reasoning replay through two parallel mechanisms in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts).

The `REASONING_REPLAY_PROVIDERS` set and `REASONING_REPLAY_MODEL_PATTERNS` regex list define eligible targets, with helper functions `isDeepSeekReasoningModel` and `requiresReasoningReplay` performing the actual classification ([lines 26-78](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts#L26-L78)).

This detection runs on every request to determine whether reasoning storage and replay logic should activate.

## The Write Path: Capturing Reasoning Content

When a tool‑calling response arrives, `cacheReasoningFromAssistantMessage` extracts the `reasoning_content` string and persists it through `cacheReasoning` or `cacheReasoningBatch`. These delegate to `cacheReasoningByKey`, which writes to both storage layers ([lines 39-80](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts#L39-L80)).

```typescript
// Capture reasoning from an assistant response
import { cacheReasoningFromAssistantMessage } from '@/open-sse/services/reasoningCache';

cacheReasoningFromAssistantMessage(
  assistantMessage,           // the raw assistant payload from upstream
  providerId,                // e.g. "deepseek"
  modelId,                   // e.g. "deepseek-v4-pro"
  { requestId, messageIndex } // context for synthetic keys when no tool calls present
);

```

Keys are generated from `tool_call.id` when available, or constructed as `request:<id>:message:<idx>` for non‑tool responses. This dual‑key strategy ensures reasoning survives regardless of response structure.

## In‑Memory Cache Configuration

The hot path uses a memory‑first design with strict resource boundaries to prevent unbounded growth:

| Parameter | Value | Purpose |
|-----------|-------|---------|
| Maximum entries | 200 | Prevents memory exhaustion under load |
| Per‑entry limit | 10 KB | Caps individual reasoning payload size |
| TTL | 2 hours | Automatic expiration of stale entries |

The `memoryCache` Map, `evictOldest` eviction policy, and `purgeExpiredMemory` cleanup run continuously on access patterns ([lines 5-20, 44-58](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts#L5-L20)).

## The Read Path: Re‑Injecting Reasoning on Subsequent Turns

Before forwarding client requests upstream, OmniRoute calls `lookupReasoning(toolCallId)` to check for cached reasoning from previous turns. The lookup follows a memory‑first, DB‑fallback pattern with automatic promotion on hit ([lines 84-108](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts#L84-L108)).

```typescript
// Re‑inject cached reasoning on the next request
import { lookupReasoning, recordReplay } from '@/open-sse/services/reasoningCache';

const cached = lookupReasoning(toolCallId);
if (cached) {
  // Satisfy the provider's reasoning_content requirement
  requestBody.reasoning_content = cached;
  recordReplay(); // increment replay counter for metrics
}

```

If found, the cached string is inserted into the outgoing request payload and the entry is promoted back to memory for future fast access. Misses are explicitly recorded for observability.

## SQLite Persistence for Cross‑Process Reliability

The database layer in [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts) provides durability across server restarts and horizontal scaling scenarios. Core operations include:

- `setReasoningCache` – write or update cached reasoning
- `getReasoningCache` – retrieve by key with optional expiration filtering
- `deleteReasoningCache` – manual invalidation
- `cleanupExpiredReasoning` – scheduled pruning of stale rows

The cleanup job at [`src/lib/jobs/reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts) runs periodically via [`src/server-init.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/server-init.ts) to maintain table health.

## Multi‑Turn Conversation Workflow

A complete reasoning replay cycle proceeds through three phases:

1. **First turn** – Model returns tool calls with `reasoning_content`. OmniRoute extracts and caches this under each `tool_call.id`.

2. **Subsequent turn** – Client sends follow‑up without `reasoning_content`. The router intercepts the request, looks up cached reasoning by tool call ID, and injects it before upstream delivery.

3. **Replay accounting** – Each injection increments the `replays` counter, enabling dashboard visibility into replay frequency and cache effectiveness.

This automation eliminates the need for clients to manually track and resubmit reasoning chains across conversation turns.

## Metrics and Observability

The service exposes operational telemetry through `getReasoningCacheServiceStats` and `getReasoningCacheServiceEntries` ([lines 39-54, 44-52](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts#L39-L54)).

```typescript
// Expose stats to the dashboard
import { getReasoningCacheServiceStats } from '@/open-sse/services/reasoningCache';

const stats = getReasoningCacheServiceStats();
// stats.hits, stats.misses, stats.replayRate, etc.

```

These power the Settings UI and administrative dashboards, showing cache hit rates and reasoning replay frequency per model.

## Key Implementation Files

| File | Responsibility |
|------|--------------|
| [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) | Core hybrid cache with memory + DB coordination |
| [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts) | SQLite schema and CRUD operations |
| [`src/lib/jobs/reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts) | Scheduled expiration of database rows |
| [`src/server-init.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/server-init.ts) | Job initialization on server startup |
| [`tests/unit/service-reasoning-cache.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/service-reasoning-cache.test.ts) | Unit tests for cache behavior |
| [`tests/unit/reasoning-replay-big-pickle.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/reasoning-replay-big-pickle.test.ts) | Integration tests for complex multi‑turn scenarios |

## Summary

- **Hybrid storage** combines in‑memory speed (200 entries, 2‑hour TTL) with SQLite durability for reliable reasoning replay across server restarts.

- **Automatic detection** via `requiresReasoningReplay` eliminates manual configuration for compatible providers and models.

- **Transparent injection** on the read path satisfies upstream API contracts without client code changes.

- **Full observability** through hit/miss/replay metrics enables operational monitoring of cache performance.

## Frequently Asked Questions

### What happens if reasoning content exceeds the 10 KB per‑entry limit?

Entries larger than 10 KB are rejected from the in‑memory cache but may still persist to SQLite depending on database configuration. The service prioritizes availability over strict size enforcement for critical reasoning chains.

### Does reasoning replay work for non‑tool‑calling responses?

Yes. When no `tool_call.id` exists, OmniRoute generates synthetic keys using `request:<id>:message:<idx>` format, enabling reasoning replay for standard assistant messages that expose thinking content.

### How does OmniRoute handle cache expiration mid‑conversation?

Expired entries are purged from memory on access and cleaned from SQLite by the periodic job. If a lookup occurs after expiration, the cache miss triggers normal request forwarding without reasoning injection—clients may receive a provider error in strict‑mode configurations.

### Can multiple OmniRoute instances share reasoning cache state?

Yes. The SQLite persistence layer enables cross‑process sharing when instances connect to the same database. In‑memory caches remain instance‑local, with the database serving as the consistent source of truth for reasoning replay across deployments.