# How OmniRoute's Session Manager Maintains State Across Retries

> Discover how OmniRoute's session manager maintains state across retries using in-memory maps and session fingerprints for persistent tool timing, sticky routing, and API key throttling. Learn more.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: internals
- Published: 2026-07-07

---

**OmniRoute maintains per-session state across retries using an in-memory map keyed by deterministic session fingerprints, enabling persistent tool timing, sticky routing, and per-API-key throttling throughout the HTTP request lifecycle.**

OmniRoute implements a lightweight session management layer that survives the lifetime of HTTP requests to handle conversation retries gracefully. The system stores session metadata in memory, allowing subsequent retries of the same logical conversation to access previous context without persisting data to disk. This article examines how the [`sessionManager.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/sessionManager.ts) service implements deterministic session identification, state updates, and lifecycle management to maintain continuity across retries.

## Session Fingerprinting and Deterministic ID Generation

OmniRoute generates stable session identifiers using the `generateSessionId` function defined in [`open-sse/services/sessionManager.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/sessionManager.ts) (lines 103-146). This function creates a deterministic hash derived from the request payload, including the model name, optional provider, system prompt, first user message, tool signatures, and an optional `connectionId` header.

Because the hash is deterministic, identical conversation payloads always produce the same session ID. This ensures that when a client retries a request, the system recognizes it as a continuation of the existing session rather than a new conversation.

```typescript
import { generateSessionId } from '@/open-sse/services/sessionManager';

const body = req.json();               // request payload
const sessionId = generateSessionId(body, {
  provider: 'openai',
  connectionId: req.headers.get('x-connection-id'),
});
// → deterministic 16-byte hex string, reused on retries

```

## Touch Session Pattern for State Persistence

Once generated, the session ID is passed to `touchSession` (lines 150-164), which implements an upsert pattern. If the session exists, the function updates the `lastActive` timestamp and increments the `requestCount`. If the session is new, it initializes a fresh entry in the in-memory map.

Retrieval functions such as `getSessionInfo`, `getSessionConnection`, and `getActiveSessionCount` (lines 90-122) provide read-only access to this metadata for routing logic without exposing the underlying Map structure.

```typescript
import { touchSession } from '@/open-sse/services/sessionManager';

function handleRequest(req) {
  const sessionId = generateSessionId(req.body, { provider: 'anthropic' });
  touchSession(sessionId, req.headers.get('x-connection-id'));
  // … routing logic …
}

```

## Tracking Tool Execution Across Retries

The session manager preserves tool execution timestamps across request boundaries using `markToolFinish` and `consumeToolFinishTime` (lines 168-188). When a tool call completes, `markToolFinish` stores the current timestamp on the session object. On the subsequent retry, `consumeToolFinishTime` retrieves and clears this value, enabling accurate calculation of "time-to-first-tool" metrics that span multiple HTTP requests.

```typescript
import { markToolFinish, consumeToolFinishTime } from '@/open-sse/services/sessionManager';

// After a tool call finishes:
markToolFinish(sessionId);

// On the next request (retry) we can read the elapsed time:
const toolFinishTs = consumeToolFinishTime(sessionId);
if (toolFinishTs) {
  const elapsedMs = Date.now() - toolFinishTs;
  console.log(`TTFT across retry: ${elapsedMs} ms`);
}

```

## Session Lifecycle and Cleanup

Session state persists for a configurable duration via a TTL mechanism implemented in [`open-sse/services/sessionManager.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/sessionManager.ts) (lines 44-82). A periodic timer executes every minute, evicting sessions whose `lastActive` timestamp exceeds 15 minutes (configurable via `SESSION_TTL_MS`). The system also enforces a hard cap of 200 concurrent sessions, automatically removing the oldest entries when this limit is exceeded.

This cleanup strategy prevents memory leaks while ensuring that active retry sequences remain unaffected during brief interruption windows.

## Enforcing Per-API-Key Limits

To prevent quota bypasses during retries, OmniRoute tracks session ownership through a secondary `Map<string, Set<string>>` called `activeSessionsByKey`. This structure maps API keys to their active session sets, enabling the `checkSessionLimit`, `registerKeySession`, and `unregisterKeySession` functions (lines 41-99) to enforce per-key session quotas.

When a retry arrives, the system checks `activeSessionsByKey` to determine whether the API key has exceeded its allocated session count, ensuring retries consume existing slots rather than creating unauthorized new sessions.

```typescript
import {
  checkSessionLimit,
  registerKeySession,
  unregisterKeySession,
} from '@/open-sse/services/sessionManager';

function maybeCreateSession(apiKeyId, sessionId, maxSessions) {
  const limitError = checkSessionLimit(apiKeyId, maxSessions);
  if (limitError) {
    return { error: limitError };
  }
  registerKeySession(apiKeyId, sessionId);
  // …
}

// When the request ends (or TTL expires):
unregisterKeySession(apiKeyId, sessionId);

```

## Summary

- OmniRoute uses **deterministic session fingerprinting** via `generateSessionId` to identify conversation retries consistently across multiple HTTP requests.
- The **touch session pattern** updates `lastActive` timestamps and request counts, keeping session entries alive during active retry sequences.
- **Tool timing persistence** through `markToolFinish` and `consumeToolFinishTime` enables accurate performance metrics that span request boundaries.
- **TTL cleanup** runs every minute with a 15-minute expiration window and a 200-session hard cap to prevent memory exhaustion.
- **Per-API-key tracking** via `activeSessionsByKey` ensures retries respect quota limits rather than spawning unauthorized new sessions.

## Frequently Asked Questions

### How does OmniRoute identify the same conversation across retries?

OmniRoute computes a deterministic session ID using `generateSessionId` in [`open-sse/services/sessionManager.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/sessionManager.ts) (lines 103-146), which hashes the model name, provider, system prompt, first user message, tool signatures, and optional connection ID. Because identical payloads produce identical hashes, retries automatically map to their existing session state.

### What happens to session state when the 15-minute TTL expires?

When a session's `lastActive` timestamp exceeds the `SESSION_TTL_MS` threshold (default 15 minutes), the periodic cleanup timer removes the entry from the in-memory map and calls `unregisterKeySession` to release the API key allocation. Subsequent retries for that conversation generate a new session ID and start fresh state accumulation.

### How does the session manager prevent API key quota bypasses during retries?

The system maintains an `activeSessionsByKey` Map that tracks all sessions associated with each API key. Before creating a new session, `checkSessionLimit` (lines 41-99) verifies whether the key has reached its maximum allowed sessions. Since retries use the same session ID generated by `generateSessionId`, they do not trigger new registrations and therefore cannot bypass the per-key limit.

### Where is session state stored in OmniRoute?

Session state resides in an in-memory Map structure within the Node.js process running [`open-sse/services/sessionManager.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/sessionManager.ts). The system does not persist session data to disk or external databases for standard retry scenarios, relying instead on the TTL-based cleanup mechanism to manage memory usage. For long-lived affinity requirements, the separate [`src/lib/db/sessionAccountAffinity.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/sessionAccountAffinity.ts) layer handles database persistence.