# How OmniRoute's Reasoning Cache Works for Multi-Turn Conversations with Strict Providers

> Discover how OmniRoute's reasoning cache powers smooth multi-turn conversations with strict providers like DeepSeek. Learn about its three-layer replay cache mechanism.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-07-10

---

**OmniRoute implements a three-layer Reasoning Replay Cache—combining an in-memory LRU store, SQLite persistence, and a background cleanup job—to automatically capture and re-inject `reasoning_content` blocks, enabling seamless multi-turn conversations with strict thinking-mode providers like DeepSeek and Kimi.**

OmniRoute acts as a unified gateway for diverse LLM providers, but "thinking-mode" models such as DeepSeek V4, Kimi K2, and Qwen-Thinking enforce strict requirements: they return a `reasoning_content` block that must be echoed back in subsequent requests or the upstream returns a 400 error. The reasoning cache eliminates this friction by storing assistant reasoning persistently and replaying it automatically without client-side management.

## The Three-Layer Cache Architecture

OmniRoute’s reasoning cache operates through a tiered storage system designed for speed, durability, and automatic maintenance.

### In-Memory LRU Store

The fastest layer is a capped in-memory map living in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts). This LRU-style cache holds up to **2,000 entries** and provides immediate lookup for the current process. It serves as the first check during request translation, minimizing latency for active conversations.

### SQLite Persistence

For durability across restarts, the system writes every entry to [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts). This module manages a SQLite table storing `tool_call_id → reasoning_content` mappings along with metadata including `provider`, `model`, and an `expires_at` timestamp (default **24 hours**). This layer ensures that even if the OmniRoute process restarts, multi-turn conversations can continue without losing reasoning context.

### Background Cleanup Job

To prevent unbounded growth, [`src/lib/jobs/reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts) runs a periodic sweep. It respects the `OMNIROUTE_REASONING_CACHE_CLEANUP_INTERVAL_MS` environment variable (defaulting to **30 minutes**) to purge expired rows from both the database and in-memory structures.

## Capturing Reasoning Content

When a strict provider returns a response, OmniRoute extracts and persists the reasoning content before returning the result to the client.

In [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts), the handler `cacheReasoningFromAssistantMessage` processes each assistant message containing `tool_calls`. It extracts the `reasoning_content` field (falling back to the legacy `reasoning` field for compatibility) and persists it via `setReasoningCache`. This function atomically updates both the in-memory map and the SQLite table, recording the raw reasoning text alongside the `toolCallId`, provider, model, and expiration timestamp.

## Replaying Content for Strict Providers

During request translation, OmniRoute determines whether the target provider requires reasoning replay and hydrates the outgoing payload accordingly.

The helper `requiresReasoningReplay` in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) checks hard-coded allow-lists (the `REASONING_REPLAY_PROVIDERS` set) to identify strict models. If matched, the system iterates through prior assistant messages and calls `lookupReasoning` for each `tool_call_id`:

```typescript
if (requiresReasoningReplay({ provider, model })) {
    const cached = lookupReasoning(params);
    if (cached) {
        msg.reasoning_content = cached;
        recordReplay(); // Bump hit statistics
    } else {
        msg.reasoning_content = ""; // Prevent 400 errors on miss
    }
}

```

- **Cache hits** inject the exact stored reasoning block.
- **Cache misses** populate the field with an empty string to maintain API contract validity, avoiding the "Param Incorrect" error.
- **Non-strict providers** bypass the cache entirely, with `reasoning_content` stripped to avoid quadratic token growth.

## Configuration and Observability

OmniRoute exposes management endpoints and environment variables to monitor and control the cache behavior.

### Environment Variables

- `OMNIROUTE_REASONING_CACHE_CLEANUP_INTERVAL_MS`: Controls the cleanup job frequency (default: 1800000ms/30min).
- Default TTL: 24 hours for entries before automatic expiration.

### Management API Endpoints

**Inspect cache status:**

```bash
curl "https://your-omniroute.local/api/cache/reasoning"

```

Returns JSON with hit/miss counters, entry counts, and recent entries:

```json
{
  "hits": 124,
  "misses": 7,
  "entries": [
    {
      "toolCallId": "call_abc123",
      "provider": "deepseek",
      "model": "deepseek-v4",
      "reasoning": "... captured reasoning ..."
    }
  ]
}

```

**Clear cache by provider:**

```bash
curl -X DELETE "https://your-omniroute.local/api/cache/reasoning?provider=deepseek"

```

## Implementation Examples

### Adding a New Strict Provider

To enable reasoning replay for additional providers, extend the allow-list in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts):

```typescript
const REASONING_REPLAY_PROVIDERS = new Set([
  'deepseek/v4',
  'kimi/k2',
  'qwen/thinking',
  'newstrict/provider', // Add here
]);

```

### Manual Cache Lookup in Custom Handlers

For debugging or custom logic, import the lookup utilities directly:

```typescript
import { requiresReasoningReplay, lookupReasoning } from '@/open-sse/services/reasoningCache';

async function translateRequest(body, provider, model) {
  if (requiresReasoningReplay({ provider, model })) {
    for (const msg of body.messages) {
      if (msg.role === 'assistant' && msg.tool_calls) {
        for (const tc of msg.tool_calls) {
          const cached = await lookupReasoning({
            provider,
            model,
            toolCallId: tc.id,
          });
          if (cached) {
            tc.reasoning_content = cached;
          }
        }
      }
    }
  }
}

```

## Summary

- OmniRoute uses a **three-layer architecture** (in-memory LRU, SQLite, cleanup job) to manage reasoning content for strict providers.
- `cacheReasoningFromAssistantMessage` captures reasoning from assistant responses in [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts).
- `requiresReasoningReplay` and `lookupReasoning` automatically inject cached content into subsequent requests to prevent 400 errors.
- Entries persist for **24 hours** by default, with a configurable cleanup interval (default **30 minutes**).
- The cache exposes REST endpoints at `/api/cache/reasoning` for inspection and manual purging.

## Frequently Asked Questions

### What happens if the reasoning cache misses for a strict provider?

If `lookupReasoning` returns null for a strict provider, OmniRoute injects an empty string (`""`) into the `reasoning_content` field rather than omitting it. This prevents the upstream provider from returning a 400 "Param Incorrect" error, though the conversation may lack historical reasoning context for that specific tool call.

### How long does OmniRoute retain reasoning content?

By default, reasoning cache entries expire after **24 hours**. This TTL is configurable in the database schema, and a background cleanup job runs every **30 minutes** (configurable via `OMNIROUTE_REASONING_CACHE_CLEANUP_INTERVAL_MS`) to remove expired rows from both SQLite and the in-memory store.

### Which providers require reasoning replay in OmniRoute?

The current implementation maintains hard-coded allow-lists in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) including models like **DeepSeek V4**, **Kimi K2**, and **Qwen-Thinking**. The `requiresReasoningReplay` function checks against these sets to determine if a provider mandates the `reasoning_content` echo behavior.

### Can I manually inspect or clear the reasoning cache?

Yes. OmniRoute exposes REST endpoints at `/api/cache/reasoning` (defined in [`src/app/api/cache/reasoning/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/cache/reasoning/route.ts)). Send a **GET** request to retrieve hit/miss statistics and recent entries, or a **DELETE** request with optional `provider` or `toolCallId` filters to clear specific cached reasoning blocks.