How Reasoning Replay Reinject Thinking Content for Multi-Turn Sessions in OmniRoute

OmniRoute uses a hybrid in-memory and SQLite cache to capture reasoning_content from assistant messages and reinject it into subsequent API requests, satisfying strict provider contracts for "thinking-mode" models like DeepSeek V4 and Kimi Coding.

The OmniRoute proxy layer enables seamless multi-turn conversations with reasoning-capable LLMs by implementing a Reasoning Replay Cache service. This system captures the model's internal reasoning trace, stores it keyed by tool-call ID, and transparently reappends it to future requests—ensuring providers that enforce stateful reasoning contracts receive the required context on every turn.

Why Reasoning Replay Matters for Thinking-Mode Models

Modern reasoning models (DeepSeek V4, Kimi Coding, Xiaomi MiMo) expose their chain-of-thought through a reasoning_content field. These providers require that every subsequent request in a multi-turn conversation include the original reasoning content, or they reject the call with a "Param Incorrect" error. OmniRoute solves this by abstracting the stateful requirement into a stateless HTTP-compatible layer, allowing each turn to arrive as a fresh request while the proxy internally manages the reasoning continuity.

The Three-Phase Reasoning Replay Flow

The replay mechanism operates through a precise sequence of capture, lookup, and injection managed across the cache service and request translators.

Capture and Cache the Reasoning Content

When an upstream response arrives containing assistant output, the cacheReasoningFromAssistantMessage function in open-sse/services/reasoningCache.ts extracts the reasoning string and persists it. The function checks for both the modern reasoning_content field and the legacy reasoning field as a fallback.

// open-sse/services/reasoningCache.ts (lines 255-259)
const reasoning =
  typeof message.reasoning_content === "string" && message.reasoning_content.length > 0
    ? message.reasoning_content
    : typeof message.reasoning === "string" && message.reasoning.length > 0
      ? message.reasoning
      : undefined;
if (!reasoning) return 0;

// Store for each tool_call_id (lines 281-289)
cacheReasoningBatch(toolCallIds, provider, model, reasoning);

The system stores entries in both a hot in-memory Map and the SQLite reasoning_cache table, associating the content with specific tool_call_id values for precise retrieval.

Lookup Previously Stored Reasoning

Before building the outbound request payload, the translator queries the cache using lookupReasoning, which checks the memory layer first for O(1) performance, then falls back to the database if necessary.

// open-sse/services/reasoningCache.ts (lines 292-298)
export function lookupReasoning(toolCallId: string): string | null {
  const mem = memoryCache.get(toolCallId);
  if (mem) return mem.reasoning;
  const dbResult = getReasoningCache(toolCallId);
  return dbResult?.reasoning ?? null;
}

Reinject Content into Outgoing Requests

The translator merges the cached content into the request body before execution. In open-sse/translator/index.ts around line 494, the system retrieves the reasoning for the current tool-use ID and assigns it to the reasoning_content field of the outgoing payload.

// open-sse/translator/index.ts (excerpt around line 494)
const cached = lookupReasoning(firstToolUseId);
if (cached) {
  // inject into the request payload
  request.body.reasoning_content = cached;
}

Provider-specific handlers like those in open-sse/translator/helpers/claudeHelper.ts extend this pattern with additional telemetry, recording when a replay occurs to track cache efficacy.

// open-sse/translator/helpers/claudeHelper.ts (lines 546-553)
const cached = lookupReasoning(pairedToolUseId);
if (cached) {
  request.body.reasoning_content = cached;
  recordReplay();   // mark that a replay occurred
}

Implementation Deep Dive

The reasoning replay system spans multiple architectural layers to balance speed with persistence.

Core Cache Service

open-sse/services/reasoningCache.ts implements the public API surface, managing the dual-layer storage strategy. The in-memory Map provides immediate access for active conversations, while the database layer ensures reasoning survives service restarts. Each entry includes a TTL (time-to-live) of 30 minutes by default.

Database Persistence Layer

src/lib/db/reasoningCache.ts defines the SQLite schema and helper functions (setReasoningCache, getReasoningCache). This layer ensures that even if the in-memory cache is cleared, the reasoning content remains recoverable for the duration of the TTL window.

Translation and Injection

open-sse/translator/index.ts acts as the orchestration point, determining when reasoning lookup is required based on the provider and model configuration. It handles the final mutation of the request body before the upstream executor dispatches the HTTP call.

Provider-Specific Handling

open-sse/translator/helpers/claudeHelper.ts demonstrates how individual provider adapters can extend the base replay logic. This file includes the recordReplay() call, which marks the conversation metadata to indicate that cached reasoning was successfully utilized for that turn.

Memory Management and Cleanup

To prevent unbounded growth of cache entries, OmniRoute runs a periodic cleanup job defined in src/lib/jobs/reasoningCacheCleanupJob.ts. This job scans both the in-memory structures and the SQLite database, evicting entries that exceed the 30-minute TTL threshold. The hybrid eviction strategy ensures that memory usage stays bounded for high-throughput deployments while maintaining database hygiene.

Code Example: Implementing End-to-End Reasoning Replay

The following TypeScript example demonstrates the complete lifecycle from capturing reasoning after a model response to reinjecting it in the next user turn:

// 1️⃣ Capture reasoning from response (executed after upstream call)
import { cacheReasoningFromAssistantMessage } from '@omniroute/open-sse/services/reasoningCache';

await cacheReasoningFromAssistantMessage({
  requestId,
  messages: upstreamResponse.choices[0].message, // contains reasoning_content
  provider,
  model,
});

// 2️⃣ Build next request (translator step)
import { lookupReasoning } from '@omniroute/open-sse/services/reasoningCache';

function buildRequest(prevToolCallId: string, userPrompt: string) {
  const cached = lookupReasoning(prevToolCallId);
  const body: any = { 
    model, 
    messages: [{ role: 'user', content: userPrompt }] 
  };
  
  if (cached) {
    body.reasoning_content = cached;   // re-inject the cached thinking content
  }
  
  return body;
}

When the subsequent turn executes, the cached reasoning_content is transparently added to the payload, satisfying the provider's contract for continued reasoning across multi-turn sessions.

Summary

• Reasoning replay is required for thinking-mode models (DeepSeek V4, Kimi Coding) that mandate reasoning_content be present in every multi-turn request.
• OmniRoute uses a hybrid cache (in-memory Map + SQLite) keyed by tool_call_id to store and retrieve reasoning content with O(1) lookup performance.
• The cacheReasoningFromAssistantMessage function in open-sse/services/reasoningCache.ts handles extraction and persistence, while lookupReasoning enables retrieval.
• Translators in open-sse/translator/index.ts and provider helpers like open-sse/translator/helpers/claudeHelper.ts handle the final injection into request payloads.
• A 30-minute TTL and periodic cleanup job keep memory usage bounded while maintaining conversation continuity.

Frequently Asked Questions

What is reasoning replay and why is it required?

Reasoning replay is the process of capturing an AI model's internal reasoning content from one API turn and reinjecting it into subsequent requests. Providers like DeepSeek and Kimi enforce strict contracts requiring this field on every turn of a multi-turn conversation; without it, the API returns a parameter error and the conversation breaks.

How does OmniRoute store reasoning content between turns?

OmniRoute implements a dual-layer storage system in open-sse/services/reasoningCache.ts. Hot entries reside in an in-memory Map for O(1) access, while a persistent SQLite table defined in src/lib/db/reasoningCache.ts ensures reasoning survives service restarts. Each entry is keyed to a specific tool_call_id and includes provider and model metadata.

What is the default TTL for cached reasoning entries?

Cached reasoning entries expire after 30 minutes by default. The reasoningCacheCleanupJob.ts periodically evicts both in-memory and database entries exceeding this TTL, preventing memory leaks in long-running deployments while maintaining sufficient context for typical conversation durations.

Which AI providers require reasoning replay functionality?

The implementation specifically supports "thinking-mode" models including DeepSeek V4, Kimi Coding, and Xiaomi MiMo. These providers expose reasoning through the reasoning_content field (or legacy reasoning field) and require this content to be echoed back in subsequent requests to maintain conversation state.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}