# How FreeLLMAPI Implements Sticky Sessions for LLM Conversations

> Discover how FreeLLMAPI uses deterministic session keys to ensure conversation continuity by routing all turns to the same LLM instance, guaranteeing a seamless user experience.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-09-01

---

**FreeLLMAPI maintains conversation continuity by mapping a deterministic session key to the initial `model_db_id`, forcing subsequent turns to route to the same LLM instance for the duration of the session.**

FreeLLMAPI is an open-source routing layer for Large Language Models that ensures conversation consistency through **sticky sessions**. By binding a chat session to a specific model instance, the system prevents mid-conversation model switches that could degrade context awareness or introduce hallucinations. The implementation relies on an in-memory session store with TTL-based expiration, managed through a tight integration between the proxy route handler and the model router.

## Session Key Generation Strategy

The foundation of sticky session tracking begins with deterministic session key generation. In [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts), the `getSessionKey()` function generates a hash from either the first user message content or an explicit `X-Session-Id` header provided by the client. This ensures that all turns of the same conversation produce an identical lookup key, enabling the system to recognize returning sessions.

```typescript
// Located in server/src/routes/proxy.ts (lines 39-49)
export function getSessionKey(
  messages: Array<{role: string; content: string}>,
  sessionIdHeader?: string,
  routingStrategy?: string
): string {
  // Implementation derives hash from first user message or header
  // combined with optional routing strategy context
}

```

## In-Memory Sticky Session Store

The core persistence mechanism is an in-memory `Map` that associates session keys with their assigned models. Located in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts), the `stickySessionMap` stores objects containing `modelDbId` and `lastUsed` timestamps, with entries automatically expiring after **30 minutes** (the **STICKY_TTL_MS** constant).

### Retrieving Sticky Assignments

The `getStickyModel()` function (lines 52-66 in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts)) checks the map for existing session entries, validating the `lastUsed` timestamp against the TTL before returning the stored `modelDbId`. If the entry is stale or absent, the function returns `undefined`, allowing normal routing to proceed.

### Updating and Cleanup

After a successful model response, `setStickyModel()` (lines 69-73) records or refreshes the mapping. This function also handles garbage collection (lines 74-90), removing stale entries and enforcing a maximum map size of **1000 entries** to bound memory usage.

```typescript
// server/src/routes/proxy.ts - Sticky session management
const stickySessionMap = new Map<string, {modelDbId: number; lastUsed: number}>();
const STICKY_TTL_MS = 30 * 60 * 1000; // 30 minutes
const MAX_STICKY_ENTRIES = 1000;

export function getStickyModel(
  messages: Array<{role: string; content: string}>,
  sessionIdHeader?: string
): number | undefined {
  const key = getSessionKey(messages, sessionIdHeader);
  const entry = stickySessionMap.get(key);
  if (entry && Date.now() - entry.lastUsed < STICKY_TTL_MS) {
    return entry.modelDbId;
  }
  return undefined;
}

export function setStickyModel(
  messages: Array<{role: string; content: string}>,
  modelDbId: number,
  sessionIdHeader?: string
): void {
  const key = getSessionKey(messages, sessionIdHeader);
  stickySessionMap.set(key, {modelDbId, lastUsed: Date.now()});
  
  // Cleanup logic: remove expired entries and enforce size limit
  // (implementation lines 74-90)
}

```

## Routing Resolution and Validation

Before executing the routing strategy, FreeLLMAPI validates sticky preferences against current model availability. The `resolveStickyPreference()` function in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) (lines 2171-2177) verifies whether the stored model remains enabled in the active fallback chain. If the model has been disabled or removed from the chain, the sticky preference is discarded to prevent routing failures.

```typescript
// server/src/services/router.ts (lines 2171-2177)
function resolveStickyPreference(stickyModelId?: number): number | undefined {
  if (!stickyModelId) return undefined;
  
  // Verify the sticky model exists in current active chain
  const isEnabled = activeChain.some(model => model.id === stickyModelId);
  return isEnabled ? stickyModelId : undefined;
}

```

## Complete Session Lifecycle Flow

Each request traverses a specific pipeline to maintain sticky session integrity:

1. **Key Generation**: The proxy route handler computes a session identifier using `getSessionKey()` from either message content or the `X-Session-Id` header.

2. **Lookup**: The system queries `getStickyModel()` to retrieve any existing model assignment for this conversation.

3. **Validation**: The retrieved ID passes to `resolveStickyPreference()` in the router service to confirm the model remains active in the current fallback chain.

4. **Routing**: If valid, the sticky model receives priority placement at the front of the candidate chain; otherwise, the router proceeds with standard strategy-based selection (priority, bandit, etc.).

5. **Persistence**: Upon successful model completion, `setStickyModel()` refreshes the mapping and TTL, ensuring continuity for the next turn.

This design is **strategy-independent**: the sticky session logic operates outside the routing strategy implementation, ensuring consistent behavior whether using priority routing, multi-armed bandit selection, or custom load balancing.

## practical Implementation Examples

To manually interact with the sticky session system, import the utility functions from the proxy route module:

```typescript
import { getSessionKey, getStickyModel, setStickyModel } from './routes/proxy.js';

// 1. Generate a session key from conversation context
const sessionKey = getSessionKey(messages, req.headers['x-session-id']);

// 2. Check for an existing sticky assignment
const priorModelId = getStickyModel(messages, req.headers['x-session-id']);

// 3. After routing selects model 42, lock it to this session
setStickyModel(messages, 42, req.headers['x-session-id']);

```

Typical integration within the request handling pipeline follows this pattern:

```typescript
// Inside the proxy route handler (server/src/routes/proxy.ts)
const sessionKey = getSessionKey(messages, req.headers['x-session-id']);
const priorModel = getStickyModel(messages, req.headers['x-session-id']);

let preferredModel = resolveStickyPreference(priorModel);
if (!preferredModel) {
  // No sticky model available or no longer enabled → normal routing
  const { chain } = resolveRoutingChain('auto');
  // Proceed with standard model selection from chain...
}

```

## Summary

- **Deterministic Session Keys**: FreeLLMAPI generates consistent session identifiers from the first user message or the `X-Session-Id` header, ensuring all conversation turns map to the same storage slot.
- **TTL-Based Storage**: The `stickySessionMap` in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) maintains assignments with a 30-minute expiration window and enforces a hard limit of 1000 concurrent sticky entries.
- **Availability Validation**: The `resolveStickyPreference()` function in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) verifies sticky models remain active in the current fallback chain before forcing selection.
- **Strategy Independence**: Sticky session enforcement operates as a routing layer concern, functioning correctly across priority, bandit, and manual routing strategies.

## Frequently Asked Questions

### How long does FreeLLMAPI maintain a sticky session assignment?

FreeLLMAPI retains sticky session mappings for **30 minutes** (defined by the **STICKY_TTL_MS** constant in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts)). After this period of inactivity, the entry is purged during the next cleanup cycle, and subsequent requests will trigger fresh model selection from the available pool.

### Can clients override the automatic session key generation?

Yes. While FreeLLMAPI automatically derives session keys from the first user message content, clients can provide explicit session identifiers via the **`X-Session-Id`** header. The `getSessionKey()` function prioritizes this header value when present, allowing client-controlled session boundaries independent of message content hashing.

### What happens if the sticky model becomes unavailable mid-conversation?

If a previously assigned model is disabled or removed from the active fallback chain, the `resolveStickyPreference()` function in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) detects the mismatch and returns `undefined`. The router then falls back to standard strategy-based routing, selecting the next best available model from the current chain without breaking the conversation flow.

### Is the sticky session store persistent across server restarts?

No. The `stickySessionMap` is a runtime **in-memory Map** with no persistence mechanism. Server restarts or deployments clear all sticky assignments, causing conversations to resume with fresh model selection on the next request. For production deployments requiring persistence, the implementation would need modification to use Redis or a similar external store.