# How OmniRoute's Reasoning Cache Facilitates Replay of Reasoning Content for Multi-Turn Conversations

> OmniRoute's reasoning cache replays model thinking to prevent context loss in multi-turn conversations. Learn how it reuses previous reasoning for tool-calling flows.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: internals
- Published: 2026-08-04

---

**OmniRoute stores and re-injects model "thinking" so that later turns of a multi-turn conversation can reuse previously generated reasoning, preventing context loss in tool-calling flows.**

This capability is essential for providers like DeepSeek that treat reasoning as part of model state. The implementation lives in the **Reasoning Cache service** at [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) with durable storage in [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts).

---

## What Gets Cached and Why

When a provider returns a response containing **tool-call reasoning** (e.g., DeepSeek's "thinking" field), OmniRoute extracts and stores that text keyed by the tool-call ID. Each cache entry contains:

- **toolCallId**: The unique ID of the tool call generated by the model
- **provider**: Provider identifier (e.g., `deepseek`)
- **model**: Model name (e.g., `deepseek-coder-v2`)
- **reasoning**: The raw reasoning string the model produced
- **expiresAt**: TTL after which the cleanup job evicts the entry

The data lives in **two layers**: an in-memory `Map` for fast lookup and **SQLite** for durability. A periodic job at [`src/lib/jobs/reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts) removes expired rows.

---

## Writing Reasoning to the Cache

The cache is populated inside the chat handling pipeline after receiving an upstream response. In [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts), the function:

```typescript
export function cacheReasoning(
  requestId: string,
  provider: string,
  model: string,
  reasoning: string,
) { … }

```

This is called from the translator/handler when a response includes a `reasoning` field. The call records reasoning alongside the current request ID, which also serves as a correlation token for the next turn.

---

## Detecting When Replay Is Needed

When the next request in the same conversation arrives, OmniRoute checks whether **reasoning replay is required**:

```typescript
export function requiresReasoningReplay(params: {
  requestId: string;
  provider: string;
  model: string;
}) { … }

```

If the provider-model pair requires replay (e.g., DeepSeek models), the service performs a lookup:

```typescript
export function lookupReasoning(toolCallId: string): string | null { … }

```

The returned string is **injected back into the outbound request payload** so the upstream model sees the same "thinking" it generated previously.

---

## The Complete Replay Lifecycle

The reasoning cache facilitates replay through a six-step lifecycle:

1. **First turn**: Model returns a tool-call with `reasoning`
2. **Cache insert**: `cacheReasoning()` stores reasoning keyed by tool-call ID
3. **Subsequent turn**: `requiresReasoningReplay()` detects that the provider needs reasoning
4. **Lookup**: `lookupReasoning()` retrieves the cached string
5. **Injection**: Retrieved reasoning is added to the request body before forwarding upstream
6. **Cleanup**: After the configured TTL, `cleanupReasoningCache()` removes the entry

This happens transparently to users, who see only a seamless multi-turn tool-calling experience.

---

## Why Replay Matters for Multi-Turn Tool Calls

Some providers—notably DeepSeek—**treat reasoning as part of model state**. When the model issues a tool call, it may later require the same reasoning in a subsequent turn. Without replay, the model loses context, leading to inconsistent or broken tool-calling flows.

The cache guarantees that original reasoning remains available for any later request belonging to the same logical conversation.

---

## Implementation Examples

### Caching Reasoning After a Tool-Call Response

```typescript
import { cacheReasoning } from '@/open-sse/services/reasoningCache';

// After receiving a response that contains reasoning:
const requestId = ctx.requestId;       // uniquely generated per request
const provider   = 'deepseek';
const model      = 'deepseek-coder-v2';
const reasoning  = response.choices[0].message.tool_calls[0].reasoning;

await cacheReasoning(requestId, provider, model, reasoning);

```

### Conditionally Injecting Cached Reasoning

```typescript
import {
  requiresReasoningReplay,
  lookupReasoning,
} from '@/open-sse/services/reasoningCache';

async function maybeInjectReasoning(payload: any, ctx: RequestContext) {
  if (
    await requiresReasoningReplay({
      requestId: ctx.requestId,
      provider: ctx.provider,
      model: ctx.model,
    })
  ) {
    const cached = lookupReasoning(payload.tool_call_id);
    if (cached) {
      // Attach previously stored reasoning so upstream model continues its chain of thought
      payload.reasoning = cached;
    }
  }
  return payload;
}

```

### Running Cache Cleanup

```typescript
import { cleanupReasoningCache } from '@/open-sse/services/reasoningCache';

// Called by the cron job defined in src/lib/jobs/reasoningCacheCleanupJob.ts
const removed = cleanupReasoningCache(); // returns number of rows deleted
console.log(`Removed ${removed} expired reasoning cache entries`);

```

---

## Key Source Files

| File | Purpose |
|------|---------|
| [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) | Core service: cache insertion, lookup, replay detection, and cleanup |
| [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts) | SQLite-backed persistence layer |
| [`src/lib/jobs/reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts) | Scheduled job for evicting stale entries |
| [`tests/unit/reasoning-cache.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/unit/reasoning-cache.test.ts) | Unit tests verifying caching, lookup, and replay behavior |
| [`src/lib/localDb.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/localDb.ts) | Exports `ReasoningCacheEntry` types to the codebase |

As implemented in diegosouzapw/OmniRoute, these files constitute the complete Reasoning Replay Cache system.

---

## Summary

- **Dual-layer storage**: In-memory Map for speed, SQLite for durability
- **TTL-based eviction**: Automatic cleanup via scheduled job at [`src/lib/jobs/reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts)
- **Provider-aware replay**: `requiresReasoningReplay()` determines whether a given provider-model combination needs reasoning injection
- **Tool-call ID as key**: Enables precise retrieval of reasoning for specific tool invocations
- **Transparent operation**: Users experience seamless multi-turn conversations without manual intervention

---

## Frequently Asked Questions

### How does OmniRoute know which conversations need reasoning replay?

OmniRoute calls `requiresReasoningReplay()` with the request ID, provider, and model. According to the OmniRoute source code, this function checks whether the provider-model pair (e.g., DeepSeek models) is configured to require reasoning replay. If so, the system attempts to retrieve cached reasoning for any tool-call IDs present in the request.

### What happens if cached reasoning expires before the next turn?

If `lookupReasoning()` returns `null` because the entry was evicted by `cleanupReasoningCache()`, the request proceeds without injected reasoning. The model may regenerate its reasoning chain, though this can produce slightly different behavior than continuing from the original thinking. The default TTL is configurable to balance storage costs against typical conversation durations.

### Is the reasoning cache shared across multiple OmniRoute instances?

No. The cache uses **SQLite in [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts)** for persistence, but this is typically a local database file per instance. For horizontal scaling, each instance maintains its own cache; reasoning replay only works reliably when subsequent turns route to the same instance. Production deployments may need sticky session configuration or a shared Redis layer for true distribution.

### Can I disable reasoning caching entirely?

The caching behavior is integral to correct operation for providers that require replay. However, you can control **cleanup frequency** by modifying the cron schedule in [`src/lib/jobs/reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts) and **TTL duration** through environment configuration. Removing the cache entirely would break multi-turn tool calls for affected providers.