# How to Implement Reasoning Replay for Multi-Turn Conversations in OmniRoute

> Implement reasoning replay for multi-turn conversations in OmniRoute. Cache model reasoning steps in SQLite and inject them into subsequent turns via the reasoningCache service layer.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-07-04

---

**OmniRoute implements reasoning replay by caching model reasoning steps in SQLite keyed by tool-call IDs, then injecting cached entries into subsequent conversation turns via the reasoningCache service layer.**

OmniRoute is an open-source routing platform that supports 237 AI providers and includes a sophisticated **reasoning replay** mechanism for multi-turn conversations. This feature stores intermediate reasoning steps in a lightweight SQLite cache, allowing subsequent conversation turns to reference and reuse previous computational results without incurring additional costs or latency.

## Understanding the Reasoning Cache Architecture

The implementation follows a three-layer pattern that separates data persistence from business logic and API exposure.

### Database Layer in src/lib/db/reasoningCache.ts

The foundation resides in [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts), which defines the `ReasoningCacheEntry` schema and provides CRUD helpers including `setReasoningCache`, `getReasoningCache`, `deleteReasoningCache`, `clearAllReasoningCache`, `cleanupExpiredReasoning`, `getReasoningCacheStats`, and `getReasoningCacheEntries`. Each entry stores the **tool-call ID**, provider, model, reasoning payload, and expiration timestamp.

### Service Layer in open-sse/services/reasoningCache.ts

The business logic lives in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts), which orchestrates when to write new entries or retrieve existing ones. This service integrates with the request pipeline to intercept reasoning data after model completion and inject cached reasoning before request translation.

### API and Maintenance Components

The HTTP interface in [`src/app/api/cache/reasoning/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/cache/reasoning/route.ts) exposes GET, POST, and DELETE endpoints for manual cache management. Background maintenance runs through [`src/lib/jobs/reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/jobs/reasoningCacheCleanupJob.ts), initialized in [`src/server-init.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/server-init.ts), which periodically evicts expired entries based on a configurable TTL (default 24 hours).

## Storing Reasoning Results from the First Turn

When a model completes its initial reasoning step, the handler in [`open-sse/handlers/chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/handlers/chatCore.ts) extracts the tool-call ID from request metadata and persists the reasoning object. This captures the full computational context for later replay.

```typescript
// From open-sse/services/reasoningCache.ts
import { setReasoningCache } from '@/lib/db/reasoningCache';

export async function cacheReasoningIfNeeded(req, resp) {
  const toolCallId = req?.metadata?.toolCallId;
  if (toolCallId && resp?.reasoning) {
    await setReasoningCache({
      toolCallId,
      provider: req.provider,
      model: req.model,
      reasoning: resp.reasoning,
      expiresAt: Date.now() + REASONING_TTL_MS, // Default: 24 hours
    });
  }
}

```

The `REASONING_TTL_MS` constant defines the entry lifetime, after which `cleanupExpiredReasoning` removes stale data during the next cleanup cycle.

## Replaying Cached Reasoning in Subsequent Turns

To reuse reasoning in a later turn, the client includes a `reasoning` field containing the original tool-call ID. The service layer intercepts this request and substitutes the cached payload before the request reaches the upstream provider.

```typescript
import { getReasoningCache } from '@/lib/db/reasoningCache';

export async function injectCachedReasoning(req) {
  const cachedId = req.body?.reasoning?.toolCallId;
  if (!cachedId) return req;

  const entry = await getReasoningCache(cachedId);
  if (!entry) return req; // Cache miss falls back to fresh computation

  return {
    ...req,
    body: {
      ...req.body,
      reasoning: entry.reasoning, // Deterministic replay
    },
  };
}

```

Because cache entries are immutable, this replay mechanism produces **deterministic** results identical to the original computation.

## Managing and Monitoring the Cache

### HTTP API Endpoints

The REST API in [`src/app/api/cache/reasoning/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/cache/reasoning/route.ts) supports manual operations:

```bash

# List all cached entries

GET /api/cache/reasoning?token=<api-key>

# Manually insert or update an entry

POST /api/cache/reasoning?token=<api-key>
{
  "toolCallId": "abc123",
  "reasoning": { "effort": "high", "summary": "auto" }
}

# Delete specific entry

DELETE /api/cache/reasoning/<toolCallId>?token=<api-key>

```

### Dashboard Metrics

The React component `src/app/(dashboard)/dashboard/cache/components/ReasoningCacheTab.tsx` displays operational metrics including `reasoningReplayRate` (percentage of turns using cached reasoning) and `reasoningReplays` (total successful cache hits). These statistics derive from `getReasoningCacheStats()` and provide visibility into cache efficiency across all 237 supported providers.

## Complete Multi-Turn Conversation Flow

A typical multi-turn conversation follows this sequence:

1. **Turn 1**: The client sends an initial request without a tool-call ID. The model computes fresh reasoning, and the service stores the result via `setReasoningCache`.

2. **Turn 2**: The client references the previous turn by sending `reasoning: {toolCallId: "<id-from-turn-1>"}`. The service retrieves the entry via `getReasoningCache` and injects it into the request, bypassing fresh computation.

3. **Turn N**: Any subsequent turn can reference the original or a different tool-call ID, enabling **multi-turn reasoning replay** without additional compute costs or provider latency.

This flow works uniformly across all providers because the cache abstracts provider-specific reasoning formats into a standardized `ReasoningCacheEntry` structure.

## Summary

- OmniRoute stores reasoning steps in SQLite via [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts) using tool-call IDs as primary keys.
- The service layer in [`open-sse/services/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/reasoningCache.ts) handles automatic caching after model completion and injection before request translation.
- Clients trigger replay by including `reasoning.toolCallId` in subsequent requests, enabling deterministic multi-turn conversations.
- A configurable TTL (default 24 hours) and background cleanup job prevent unbounded storage growth.
- The dashboard in [`ReasoningCacheTab.tsx`](https://github.com/diegosouzapw/OmniRoute/blob/main/ReasoningCacheTab.tsx) tracks `reasoningReplayRate` and `reasoningReplays` to monitor cache efficiency.

## Frequently Asked Questions

### What is the default TTL for cached reasoning entries?

Cached reasoning entries expire after 24 hours by default. The `cleanupExpiredReasoning` function in [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts) removes stale entries, triggered by the [`reasoningCacheCleanupJob.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/reasoningCacheCleanupJob.ts) scheduler initialized in [`src/server-init.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/server-init.ts).

### How does OmniRoute handle cache misses during replay?

If `getReasoningCache` returns no entry for a requested tool-call ID, the service layer passes the original request through unchanged. The upstream provider then computes fresh reasoning, and the new result gets cached for future turns if a tool-call ID is present.

### Can reasoning be shared across different AI providers?

Yes. The cache schema in [`src/lib/db/reasoningCache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/reasoningCache.ts) stores the provider and model alongside each entry. While the cache can retrieve entries regardless of provider, the reasoning payload format remains compatible because OmniRoute normalizes reasoning structures across its 237 supported providers during the translation layer.

### Where can I view reasoning replay statistics?

The dashboard component [`ReasoningCacheTab.tsx`](https://github.com/diegosouzapw/OmniRoute/blob/main/ReasoningCacheTab.tsx) displays real-time metrics including `reasoningReplayRate` (percentage of requests using cached reasoning) and `reasoningReplays` (total hit count). These values calculate from `getReasoningCacheStats()` and update through the dashboard cache tab interface.