# How OmniRoute's Reasoning Replay Feature Works and Caches Reasoning Traces

> Explore OmniRoute's reasoning replay: learn how it caches reasoning traces for deterministic debugging and audit trails. Understand its hybrid in-memory/SQLite cache and encrypted replay capabilities.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-16

---

**OmniRoute's reasoning replay captures a model's internal chain-of-thought during multi-turn tool-calling flows and stores it in a hybrid in-memory/SQLite cache for deterministic debugging, audit trails, and optional encrypted replay.**

The **reasoning replay** feature in the `diegosouzapw/OmniRoute` codebase solves a critical problem in production LLM systems: reproducing exactly what a model was "thinking" during complex tool-calling sessions. When enabled, every reasoning step gets persisted with configurable TTL and can be injected back into response streams for downstream consumers.

## What Reasoning Replay Actually Records

Unlike standard logging, reasoning replay captures the **raw reasoning text** that models generate when deciding which tools to call. This includes the logical steps between receiving a user query and emitting a `tool_calls` or `assistant` message.

The feature operates at three architectural layers according to the source code:

| Layer | Responsibility | Key File |
|-------|---------------|----------|
| Capture | Translator inserts placeholders; stores reasoning when provider returns | [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) |
| Persistence | Hybrid in-memory + SQLite storage with durable schema | [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts) |
| Cleanup | Background job evicts expired entries from both stores | [`src/lib/jobs/reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts) |

## How Reasoning Traces Are Cached

The caching mechanism follows a four-step flow implemented across the service layer.

### 1. Scope Resolution

Each trace is keyed by a **reasoning cache scope** combining `requestId` and optional `interleaved` capability flags. The `requiresReasoningReplay` helper determines if a request qualifies for capture.

### 2. Dual-Write to Cache

The `cacheReasoning()` method in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) writes simultaneously to:

- **In-memory Map**: Fast lookup for active sessions
- **SQLite table**: Durable storage with fields `requestId`, `modelId`, `scope`, `reasoning`, `createdAt`, `expiresAt`

### 3. Cache Retrieval for Replay

When a subsequent turn requests previous reasoning, `getReasoningCache(scope)` returns the stored text. Downstream components hydrate this into the response stream as a placeholder, maintaining message ordering.

### 4. TTL-Based Expiration

Default TTL is approximately **30 minutes**. The cleanup job prunes both stores, preventing unbounded memory growth in long-running deployments.

## Enabling and Using Reasoning Replay

### API-Level Activation

Pass `reasoningReplay: true` in requests to `/api/v1/responses` (handled in [`src/app/api/v1/responses/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/v1/responses/route.ts)):

```typescript
// Enable reasoning replay for a multi-turn session
const response = await fetch("https://api.omniroute.dev/api/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "gpt-4o",
    messages: conversationHistory,
    reasoningReplay: true  // Opt-in flag
  })
});

```

### Programmatic Cache Interaction

```typescript
import { reasoningCache, ReasoningCacheScope } from "@/open-sse/services/reasoningCache";

// Define scope for this conversation
const scope: ReasoningCacheScope = {
  requestId: "chat-session-abc123",
  model: "gpt-4o",
};

// After receiving reasoning from provider
await reasoningCache.cacheReasoning(
  scope,
  "openai",           // provider
  "gpt-4o",           // model
  "Analyzing user intent →确定为 profile lookup → selecting getUserProfile tool"
);

// Later turn: retrieve and replay
const cachedReasoning = await reasoningCache.getReasoningCache(scope);
if (cachedReasoning) {
  stream.write({ role: "assistant", reasoning: cachedReasoning });
}

```

### Administrative Operations

```typescript
// Clear all cached reasoning (testing/emergency)
import { clearReasoningCacheAll } from "@/open-sse/services/reasoningCache";
await clearReasoningCacheAll();  // Wipes memory + SQLite

// Check cache health
const health = await fetch("/api/cache/reasoning").then(r => r.json());
// { entries: 42, sizeBytes: 123456, expiresSoon: 3 }

```

## Storage Implementation Details

The **SQLite schema** in [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts) implements the durable layer:

- Indexed on `(requestId, scope)` for fast scoped queries
- `expiresAt` column enables efficient range-based deletion
- Cross-process safe for horizontally-scaled deployments

The **in-memory layer** uses a Map with identical TTL semantics, providing sub-millisecond reads for hot entries while offloading cold data to SQLite.

## Configuration and Capabilities

| Setting | Control | Default |
|---------|---------|---------|
| Global toggle | `INTERLEAVED_REASONING_REPLAY` capability | Disabled |
| Per-request | `reasoningReplay` body parameter | `false` |
| Encryption | "encrypted reasoning replay" capability (PR #9876) | Disabled |
| TTL | Environment override | ~30 minutes |

## Summary

- **Reasoning replay** captures model chain-of-thought during tool-calling flows for deterministic reconstruction
- **Hybrid caching** combines in-memory speed with SQLite durability in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) and [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts)
- **Automatic cleanup** via [`reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/reasoningCacheCleanupJob.ts) prevents resource leaks with configurable TTL
- **Opt-in activation** through API flag or global capability, with encrypted variant available

## Frequently Asked Questions

### What is the reasoning replay feature in OmniRoute?

**Reasoning replay** is a debugging and audit feature that records the internal reasoning text models generate during multi-turn tool-calling conversations. It enables developers to replay exactly what a model was thinking at any point in a session, useful for troubleshooting complex agent behaviors and maintaining compliance records.

### How does OmniRoute cache reasoning traces?

OmniRoute uses a **hybrid in-memory + SQLite architecture**. The `reasoningCache` service in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) writes every trace to both a fast in-memory Map and a durable SQLite table managed by [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts). Entries include metadata like `requestId`, `modelId`, scope, and expiration timestamp for precise retrieval.

### Can reasoning replay be encrypted?

Yes. An **encrypted reasoning replay** variant exists, gated by a dedicated capability flag introduced in PR #9876. When enabled, reasoning text is encrypted at rest in SQLite and only decrypted when retrieved by authorized scopes, protecting sensitive model reasoning in production environments.

### How do I clear the reasoning cache?

Use the `clearReasoningCacheAll()` function exported from [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) to wipe both in-memory and SQLite stores. This is useful in test suites, development environments, or emergency situations requiring immediate cache invalidation.