# How Sticky Sessions Work in FreeLLMAPI: Technical Implementation Guide

> Learn how FreeLLMAPI implements sticky sessions to maintain conversation state by directing requests to the same model for up to 30 minutes. Boost performance and reliability.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-08-29

---

**FreeLLMAPI implements sticky sessions by persisting the *model* that successfully served the last assistant turn of a conversation and preferring that same model on subsequent requests for up to 30 minutes.**

FreeLLMAPI is an open-source LLM routing proxy designed to maintain conversation continuity across multiple providers. The **sticky session** mechanism ensures that multi-turn conversations remain bound to the same model, preventing context fragmentation and inconsistent responses caused by mid-conversation provider switches.

## Core Architecture of Sticky Sessions

The sticky session system consists of three tightly-coupled components that work together to maintain session stability.

### Session Key Generation

The foundation of the sticky system is deterministic session identification. In [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts), the `getSessionKey()` function constructs a unique identifier using either the client's `x-session-id` header or a SHA-1 hash derived from the first user message combined with the routing strategy key.

When present, the `x-session-id` header takes precedence, producing keys like `hdr:abc123::smart` (where "smart" represents the routing strategy). Without a header, the system generates a hash-based key from the message content and strategy, ensuring identical conversations receive consistent routing even across reconnections.

### The Sticky Map Storage

FreeLLMAPI maintains an in-memory **sticky map** defined in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) as `Map<string,{modelDbId,lastUsed}>`. This map stores the model database ID alongside a timestamp of the last successful response.

The map enforces automatic expiration through `STICKY_TTL_MS` (set to 30 minutes) and implements memory safety constraints with a soft cap of 500 entries and a hard cap of 1000. When the hard limit triggers, the system evicts the oldest entries based on the `lastUsed` timestamp.

### Router Integration

The routing layer in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) receives the sticky preference through the `preferredModelDbId` parameter in the `routeRequest()` function (lines 37-45). When a sticky model ID is present, the router either:

- Moves the existing model to the front of the sorted provider chain, or
- Injects a custom row for the model if it exists in the database but isn't part of the current active chain

This injection mechanism ensures that even custom or out-of-profile models can be honored for sticky session continuity.

## Step-by-Step Execution Flow

The sticky session mechanism operates through a six-stage pipeline for each conversation turn.

### 1. Request Intake and Key Resolution

When a request arrives at the proxy, the system extracts the optional `x-session-id` header (`sessionIdHeader`) and invokes `getSessionKey(messages, sessionIdHeader, strategyKey)` to generate the session identifier.

### 2. Sticky Lookup

The `getStickyModel()` function checks if the conversation contains an existing assistant turn (`hasAssistant`). If the session key exists in `stickySessionMap` and `Date.now() - entry.lastUsed <= STICKY_TTL_MS`, the function returns the stored `modelDbId`.

### 3. Routing Preference

The router receives `preferredModelDbId` from the sticky lookup. If present, the router locates the model in the sorted chain and splices it to index 0, ensuring highest priority selection.

### 4. Model Execution

The request dispatches to the selected model provider.

### 5. Persistence

After a successful response, the proxy calls `setStickyModel(messages, route.modelDbId, sessionIdHeader, stickyStrategyKey)` in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts). This updates the map with the current timestamp, resetting the 30-minute TTL window.

### 6. Expiration and Cleanup

On subsequent reads, expired entries (older than 30 minutes) are automatically removed from the map, causing the request to fall back to standard auto-routing logic.

## Code Implementation Examples

### Client-Side Session Initialization

```typescript
const response = await fetch('https://api.freellmapi.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'x-session-id': 'my-conversation-123',   // Sticky session identifier
  },
  body: JSON.stringify({
    model: 'auto',
    messages: [{role:'user', content:'Hello!'}],
  }),
});

```

### Proxy-Level Sticky Handling

```typescript
// Inside server/src/routes/proxy.ts
const sticky = getStickyModel(messages, sessionIdHeader, stickyStrategyKey);
const preferredModelDbId = sticky ?? undefined;

// After successful model call:
setStickyModel(messages, route.modelDbId, sessionIdHeader, stickyStrategyKey);

```

### Router Integration Logic

```typescript
// Inside server/src/services/router.ts
if (preferredModelDbId) {
  const idx = sortedChain.findIndex(e => e.model_db_id === preferredModelDbId);
  if (idx >= 0) {
    const [preferred] = sortedChain.splice(idx, 1);
    sortedChain.unshift(preferred);
  } else {
    // Inject custom/pinned model not in current chain
    const pinnedRow = db.prepare(`
      SELECT m.id as model_db_id, m.platform, m.model_id, 
             m.display_name, m.intelligence_rank
      FROM models m
      WHERE m.id = ? AND m.enabled = 1
    `).get(preferredModelDbId);
    if (pinnedRow) sortedChain.unshift(pinnedRow);
  }
}

```

## TTL and Memory Management

The sticky session system balances continuity with resource constraints through aggressive expiration policies. The **30-minute TTL** (`STICKY_TTL_MS`) prevents stale sessions from consuming memory while allowing sufficient time for typical conversational interactions.

Memory protection operates on two levels: a soft cap at 500 entries triggers background pruning of expired items, while a hard cap at 1000 entries forces eviction of the oldest `lastUsed` entries regardless of TTL status. This ensures the proxy remains stable under high traffic volumes without manual intervention.

## Summary

- **Sticky sessions** in FreeLLMAPI bind conversations to specific models using a deterministic session key and persistent map storage.
- The system lives primarily in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) with routing integration in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts).
- Session keys derive from the `x-session-id` header or message content hashes combined with routing strategy.
- Entries persist for **30 minutes** (`STICKY_TTL_MS`) with automatic cleanup via memory caps (500 soft, 1000 hard).
- The router honors sticky preferences by reordering or injecting the preferred model into the provider chain.

## Frequently Asked Questions

### What happens when a sticky session expires?

When the 30-minute TTL expires, `getStickyModel()` returns `undefined`, causing the router to ignore the stale preference and select a new model based on current availability, load, and routing strategy. The expired entry is automatically pruned from the map on the next access.

### Can clients override sticky session behavior?

Yes. Clients can force a fresh model selection by omitting the `x-session-id` header and ensuring the first user message content differs from previous turns, which generates a new session key. Alternatively, sending a new unique `x-session-id` header effectively starts a new sticky session unrelated to previous conversations.

### How does FreeLLMAPI handle sticky sessions when the preferred model becomes unavailable?

If the sticky model ID specified in `preferredModelDbId` is not found in the active chain, the router attempts to inject it by querying the database for `enabled = 1` models. If the model is disabled or deleted, the injection fails silently, and the router proceeds with the standard sorted chain, effectively falling back to auto-routing for that turn.

### What is the performance impact of sticky session lookups?

The sticky map operates as an in-memory JavaScript `Map` with O(1) lookup complexity. According to the source code in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts), the lookup involves a single `get()` operation and a timestamp comparison, adding negligible latency (microseconds) to the request processing pipeline compared to the network overhead of LLM API calls.