OmniRoute Reasoning Replay Feature: How It Handles Multi-Turn Conversations

Reasoning Replay is a hybrid in-memory and SQLite cache in OmniRoute that captures reasoning_content from providers like DeepSeek V4, Kimi K2, and Qwen-Thinking, then automatically replays it on every subsequent turn of a multi-turn conversation to prevent 400 errors.

OmniRoute's reasoning replay feature ensures strict compliance with providers that require the same reasoning_content to be passed back during tool-calling conversations. As implemented in diegosouzapw/OmniRoute, the system intercepts thinking-mode responses, persists them transparently, and reinjects them into later requests without manual client management.

What Is the Reasoning Replay Feature?

Reasoning Replay is a hybrid in-memory + SQLite cache designed to manage reasoning_content (also known as "thinking-mode") returned by certain model providers. Providers such as DeepSeek V4, Kimi K2, Qwen-Thinking, GLM, and Xiaomi MiMo enforce a strict contract: the same reasoning_content must be sent back on every subsequent turn of a tool-calling conversation. If it is missing, the provider returns a 400 error with a message like "The reasoning_content in the thinking mode must be passed back to the API."

The core service in open-sse/services/reasoningCache.ts implements the detection, caching, lookup, and cleanup routines that make this requirement invisible to API consumers.

How OmniRoute Detects Reasoning Replay Requirements

Before caching or replaying anything, the translator in open-sse/translator/index.ts must determine whether the current model requires reasoning replay. This detection happens through three ordered checks implemented in requiresReasoningReplay().

Provider and Model Whitelists

The first check evaluates hardcoded allowlists defined in [reasoningCache.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.49/open-sse/services/reasoningCache.ts#L28-L63). The constants REASONING_REPLAY_PROVIDERS and REASONING_REPLAY_MODEL_PATTERNS enumerate which upstream backends and model identifiers are known to require the reasoning-content contract.

Explicit Contract Detection

The second check inspects the interleavedField flag emitted by the upstream model. If the model exposes "reasoning_content", requiresReasoningReplay() returns true; if it exposes "reasoning_details" instead, the function returns false, because that field shape does not participate in the replay protocol.

Legacy Fallback Handling

If the model is absent from the whitelist but the allowLegacyFallback parameter is true (the default), the function falls back to the provider and model lists. This ensures newer or renamed models still receive correct handling without an immediate code change.

Caching the Reasoning Content (Write Path)

When the upstream provider returns a response containing reasoning_content, the service calls cacheReasoning() (or one of its batch helpers) defined in open-sse/services/reasoningCache.ts. The write path performs two operations:

• It stores the payload in an in-process Map for fast reads.
• It persists the identical entry to the reasoningCache SQLite table via src/lib/db/reasoningCache.ts.

This dual-layer storage guarantees that single-turn latency remains low while conversations survive process restarts.

Re-injecting Reasoning Content Into Multi-Turn Conversations (Read Path)

The read path is where the reasoning replay feature handles multi-turn conversations automatically. For every new turn of the same conversation, the following sequence executes in open-sse/translator/index.ts:

1. The translator examines the incoming request for an existing reasoning_content field.
2. If the field is missing and requiresReasoningReplay() returned true, the service executes lookupReasoning(toolCallId).
3. lookupReasoning() checks the in-memory map first, then falls back to the SQLite DB.
4. If a cached value is found, the field is re-injected into the request payload before it is sent upstream.

Because this process runs automatically, clients do not need to manually track or forward reasoning_content across multi-turn tool-calling flows.

Cache Lifecycle and Cleanup

Cached entries do not live forever. A background periodic job in src/lib/jobs/reasoningCacheCleanupJob.ts runs cleanupReasoningCache() to purge rows older than the configured TTL. For administrative tasks or test isolation, clearReasoningCacheAll() removes every entry from both the in-memory map and the SQLite table and returns the number of rows deleted.

Practical Examples

The following examples show how the reasoning replay feature works in practice.

Transparent Multi-Turn Chat Request

// Assume an OpenAI-compatible request that uses tool calls.
await fetch("https://omniroute.example.com/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Authorization": "Bearer <key>"
  },
  body: JSON.stringify({
    model: "deepseek-v4-pro",
    messages: [{ role: "user", content: "Explain the steps." }]
    // No reasoning_content – OmniRoute adds it automatically on turn 2
  })
});

On the second turn, the router uses the same conversation ID, looks up the cached reasoning_content from turn 1, and injects it automatically.

Clearing the Cache Manually

import { clearReasoningCacheAll } from "open-sse/services/reasoningCache";

// Remove all entries (e.g., after a major model upgrade)
const removed = clearReasoningCacheAll();
console.log(`Cleared ${removed} reasoning cache entries`);

Inspecting Cache Statistics

import { getReasoningCacheServiceStats } from "open-sse/services/reasoningCache";

const stats = getReasoningCacheServiceStats();
console.log("Cache entries:", stats.entries);
console.log("Memory hits:", stats.memoryHits);
console.log("DB hits:", stats.dbHits);

Summary

• Reasoning Replay is a hybrid in-memory and SQLite cache in OmniRoute that manages reasoning_content for providers requiring strict multi-turn replay contracts.
• Detection relies on REASONING_REPLAY_PROVIDERS, REASONING_REPLAY_MODEL_PATTERNS, the interleavedField flag, and an optional legacy fallback.
• The write path persists reasoning content via cacheReasoning() to both a Map and the reasoningCache table.
• The read path automatically re-injects cached content through lookupReasoning(toolCallId) on subsequent conversation turns.
• Expired entries are removed by reasoningCacheCleanupJob.ts, and clearReasoningCacheAll() supports manual or test-time purging.

Frequently Asked Questions

What providers require the reasoning replay feature?

Providers including DeepSeek V4, Kimi K2, Qwen-Thinking, GLM, and Xiaomi MiMo require the same reasoning_content to be sent back on every turn. If it is omitted, they return a 400 error stating that the reasoning content must be passed back to the API.

How does OmniRoute handle reasoning replay in multi-turn conversations?

OmniRoute caches the initial reasoning_content in memory and SQLite using cacheReasoning(). On each subsequent turn, lookupReasoning(toolCallId) retrieves the cached value and the translator in open-sse/translator/index.ts re-injects it into the outgoing request automatically.

Where is the reasoning cache physically stored?

The cache uses a dual-layer architecture: an in-process Map for low-latency reads, and a SQLite reasoningCache table managed in src/lib/db/reasoningCache.ts for durability across process restarts.

How can I manually clear or inspect the reasoning replay cache?

You can call clearReasoningCacheAll() from open-sse/services/reasoningCache to delete all entries, or call getReasoningCacheServiceStats() to inspect memory hits, DB hits, and total entry counts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}