How OmniRoute's Reasoning Replay Feature Works for Debugging

OmniRoute automatically captures and re-injects reasoning_content from thinking-mode models into subsequent API requests, preventing HTTP 400 errors while maintaining conversation state across SQLite and in-memory caches.

OmniRoute is an open-source LLM routing gateway that normalizes requests across multiple providers. When integrating with reasoning models like DeepSeek V4 or Kimi Coding, developers encounter a strict requirement: the API mandates that clients return the model's previous reasoning chain on every subsequent turn. Omitting this field triggers an HTTP 400 error with the message "The reasoning_content in the thinking mode must be passed back to the API." The reasoning replay subsystem solves this by managing reasoning state server-side, enabling seamless debugging of multi-turn conversations without client-side complexity.

Why Reasoning Replay Is Required

Thinking-mode models from providers like DeepSeek, Opencode-Go, SiliconFlow, and Kimi maintain an internal reasoning chain that must be propagated across turns. If a client fails to echo this content, the upstream provider rejects the request.

In open-sse/services/reasoningCache.ts, the requiresReasoningReplay function identifies which provider/model combinations enforce this requirement:

export function requiresReasoningReplay({ provider, model }) {
  const p = provider.trim().toLowerCase();
  const m = model.trim();
  if (REASONING_REPLAY_PROVIDERS.has(p)) return true;
  return REASONING_REPLAY_MODEL_PATTERNS.some(rx => rx.test(m));
}

This check supports specific models like deepseek-v4-flash and deepseek-v4-pro via regex patterns, ensuring the system only activates replay logic when necessary.

Capturing and Persisting Reasoning Content

When OmniRoute generates an assistant message, cacheReasoningFromAssistantMessage extracts reasoning from either reasoning_content or the legacy reasoning field:

// open-sse/services/reasoningCache.ts
export function cacheReasoningFromAssistantMessage(message, provider, model, ctx) {
  const reasoning = typeof message.reasoning_content === "string"
      ? message.reasoning_content
      : typeof message.reasoning === "string"
      ? message.reasoning
      : "";
  if (!reasoning) return 0;
  // Store under every tool_call.id, or under a synthetic request/message key
  cacheReasoningBatch(toolCallIds, provider, model, reasoning);
}

The system writes this data to two locations:

• In-memory LRU cache: For low-latency retrieval during active conversations
• SQLite table: For persistence across process restarts

The database schema, defined in migration 033_create_reasoning_cache.sql, creates the reasoning_cache table:

CREATE TABLE reasoning_cache (
  tool_call_id TEXT PRIMARY KEY,
  provider TEXT NOT NULL,
  model TEXT NOT NULL,
  reasoning TEXT NOT NULL,
  char_count INTEGER NOT NULL DEFAULT 0,
  created_at TEXT NOT NULL DEFAULT (datetime('now')),
  expires_at INTEGER NOT NULL
);

Re-injecting Reasoning During Request Translation

The core replay logic resides in open-sse/translator/index.ts. After normalizing the request from OpenAI format to the provider's native format, the translator checks isReasoner to determine if replay is required. For each assistant message in the payload, it either preserves existing reasoning_content or injects cached values:

// open-sse/translator/index.ts – reasoning replay loop
if (isReasoner && result.messages) {
  for (const [idx, msg] of result.messages.entries()) {
    if (msg.role !== "assistant") continue;
    const cacheKey = msg.tool_calls?.[0]?.id
        ?? getAssistantMessageCacheKey(result, idx);
    if (cacheKey) {
      const cached = lookupReasoning(cacheKey);
      if (cached) {
        msg.reasoning_content = cached;
        recordReplay();               // update dashboard metrics
        continue;
      }
    }
    // …fallback handling (placeholder insertion for legacy providers)
  }
}

This injection happens transparently, allowing clients to interact with reasoning models using standard OpenAI-compatible request formats while OmniRoute handles the provider-specific complexity.

Hybrid Cache Retrieval Strategy

The lookupReasoning function in open-sse/services/reasoningCache.ts implements a tiered lookup strategy:

export function lookupReasoning(toolCallId) {
  const mem = memoryCache.get(toolCallId);
  if (mem && Date.now() < mem.expiresAt) return mem.reasoning;
  const db = getReasoningCache(toolCallId);
  if (db) {
    memoryCache.set(toolCallId, { ...db, expiresAt: Date.now() + TTL_MS });
    return db.reasoning;
  }
  return null;
}

The flow follows this priority:

1. Check in-memory cache for unexpired entries
2. On miss, query the SQLite table via getReasoningCache in src/lib/db/reasoningCache.ts
3. Promote database hits back to memory cache for faster subsequent access

An auto-cleanup timer runs every 30 minutes to purge expired entries from both tiers, initiated by startAutoCleanup in open-sse/services/reasoningCache.ts:

function startAutoCleanup() {
  cleanupReasoningCache();               // one-off boot cleanup
  setInterval(() => cleanupReasoningCache(),
      getCleanupIntervalMs());           // default 30 min
}

Monitoring Cache Performance

OmniRoute exposes reasoning cache metrics through an admin REST API at src/app/api/cache/reasoning/route.ts. This endpoint reports hit rates, miss counts, and replay statistics:

# GET request – requires an admin JWT

curl -H "Authorization: Bearer $ADMIN_JWT" \
     http://localhost:20128/api/cache/reasoning

Sample response:

{
  "stats": {
    "memoryEntries": 12,
    "dbEntries": 47,
    "hits": 84,
    "misses": 6,
    "replays": 81,
    "replayRate": "90.0%"
  }
}

Operators can clear provider-specific caches during debugging:

curl -X DELETE \
     -H "Authorization: Bearer $ADMIN_JWT" \
     "http://localhost:20128/api/cache/reasoning?provider=deepseek"

Key Implementation Files

File	Purpose
open-sse/services/reasoningCache.ts	Core logic for capture, lookup, and memory management
src/lib/db/reasoningCache.ts	SQLite CRUD operations and setReasoningCache/getReasoningCache
open-sse/translator/index.ts	Request transformation and reasoning injection (lines 5110-5140)
src/lib/db/migrations/033_create_reasoning_cache.sql	Database schema definition
src/app/api/cache/reasoning/route.ts	Admin API for metrics and cache management

Summary

• OmniRoute's reasoning replay feature prevents HTTP 400 errors by ensuring reasoning_content is always present for thinking-mode providers like DeepSeek and Kimi.
• The system uses a dual-tier cache (in-memory LRU + SQLite) keyed by tool_call_id to persist reasoning across turns and process restarts.
• Provider detection in requiresReasoningReplay activates replay only for specific models matching regex patterns or provider names.
• The translator layer transparently injects cached reasoning during request transformation, allowing standard OpenAI-compatible clients to work with complex reasoning APIs.
• Automatic cleanup runs every 30 minutes, and an admin API provides visibility into hit rates and cache statistics.

Frequently Asked Questions

What causes the "reasoning_content must be passed back" error?

Upstream providers like DeepSeek and Kimi require the full reasoning chain from previous turns to maintain conversation context. When this field is missing from the request payload, the API returns HTTP 400 with the message "The reasoning_content in the thinking mode must be passed back to the API." OmniRoute intercepts these requests and injects the cached content automatically.

How does OmniRoute handle cache expiration?

Entries expire based on a configurable TTL stored in the expires_at column. The system runs cleanupReasoningCache every 30 minutes via setInterval to remove stale entries from both the in-memory map and the SQLite table. Valid entries in SQLite survive process restarts and are promoted to memory on first access.

Can I debug which reasoning content is being replayed?

Yes. The admin API at /api/cache/reasoning exposes statistics including hit counts, miss rates, and replay frequency. For granular debugging, you can inspect the reasoning_cache table directly or clear specific provider caches using the DELETE endpoint to force fresh reasoning generation on the next request.

Which models require reasoning replay?

OmniRoute activates replay for providers in REASONING_REPLAY_PROVIDERS (including DeepSeek, Opencode-Go, and SiliconFlow) and models matching patterns like /deepseek-v4-(flash|pro)/i. The requiresReasoningReplay function evaluates both the provider name and model identifier to determine if the strict reasoning contract applies.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}