# How Sticky Sessions Keep Conversations on the Same Model in FreeLLMAPI

> Learn how FreeLLMAPI's sticky sessions ensure consistent AI conversations by binding them to the same model using a cache and deterministic session keys. Prevent mid-conversation model switching.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: how-to-guide
- Published: 2026-06-22

---

**FreeLLMAPI's sticky sessions feature binds each conversation to a specific LLM model using a deterministic session key and an in-memory cache with a 30-minute TTL, ensuring consistent responses and preventing mid-conversation model switching.**

The open-source FreeLLMAPI project (`tashfeenahmed/freellmapi`) implements a lightweight sticky session mechanism that maintains conversation continuity across multiple API calls. By caching the model assignment in an in-process Map, the router can return subsequent messages to the same model that handled the initial request, avoiding the hallucinations and inconsistency that often occur when models switch mid-conversation.

## Session Key Generation

Every sticky session begins with a deterministic **session key** that identifies the conversation. In [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts), the `getSessionKey` function (lines 31-42) constructs this key using one of two strategies:

- **Explicit header**: If the client sends an `X-Session-Id` header, the router uses that value (optionally combined with a routing strategy key) as the session identifier.
- **Content hash**: When no header is present, the router generates a SHA-1 hash of the **first user message** text to create the key.

```typescript
// server/src/routes/proxy.ts
const key = getSessionKey(messages, sessionIdHeader, strategyKey);

```

This deterministic approach ensures that rerunning the same conversation without a session header still maps to the same sticky entry, while explicit headers allow clients to maintain continuity across disjointed message sequences.

## The Sticky Session Map

The core storage mechanism is a module-level `Map` defined in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) (lines 25-30):

```typescript
// server/src/routes/proxy.ts
const stickySessionMap = new Map<string, { modelDbId: number; lastUsed: number }>();
const STICKY_TTL_MS = 30 * 60 * 1000; // 30 minutes

```

Each entry stores:
- **modelDbId**: The database identifier of the LLM model that last served this session
- **lastUsed**: A timestamp tracking the last successful request

The map enforces a **hard limit of 500 entries** and a **30-minute TTL** to prevent unbounded memory growth while keeping recent sessions active.

## Reading and Writing Sticky Entries

Before routing a request, the system checks for an existing sticky assignment. The `getStickyModel` function in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) (lines 44-58) validates the TTL and returns the stored `modelDbId` only if the entry is still fresh:

```typescript
// server/src/routes/proxy.ts
const sticky = getStickyModel(messages, sessionIdHeader, stickyStrategyKey);
const preferredModel =
    (sticky != null && groupChain.some(r => r.model_db_id === sticky))
      ? sticky
      : undefined;

```

After a successful routing decision, `setStickyModel` (lines 61-72) records the model that served the request:

```typescript
// server/src/routes/proxy.ts
setStickyModel(messages, route.modelDbId, sessionIdHeader, stickyStrategyKey);

```

This write operation updates the timestamp and performs housekeeping—evicting entries that exceed the 500-entry limit or have passed the TTL threshold.

## Integration with Routing Logic

The sticky model ID flows through the routing pipeline as `preferredModelDbId`. According to the source code comments in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) (lines 95-99):

> "If preferredModelDbId is set, that model gets tried FIRST (sticky sessions). This prevents hallucination from model switching mid-conversation."

The `routeRequest` function receives this preference and prioritizes the matching model in the candidate chain. If the preferred model is unavailable or the sticky entry has expired, the router falls back to standard load-balancing logic.

## Practical Usage Examples

### Using an Explicit Session Header

Provide an `X-Session-Id` header to maintain continuity across API calls:

```typescript
import axios from 'axios';

await axios.post(
  'https://api.example.com/v1/chat/completions',
  {
    messages: [{ role: 'user', content: 'Hello, who are you?' }],
  },
  {
    headers: {
      'X-Session-Id': 'conversation-12345',
    },
  },
);

```

The first request creates a sticky entry keyed to `conversation-12345`. Subsequent calls reusing this header route to the same model until 30 minutes of inactivity passes.

### Hash-Based Sessions (No Header)

Allow the system to generate a session key from message content:

```typescript
await axios.post(
  'https://api.example.com/v1/chat/completions',
  {
    messages: [
      { role: 'user', content: 'Explain quantum tunneling.' },
    ],
  },
);

```

Without the `X-Session-Id` header, the router hashes the first user message (`"Explain quantum tunneling."`). All turns containing this identical first message share the same sticky model assignment.

### Debugging Sticky Sessions

Inspect the internal map for development purposes:

```typescript
import { stickySessionMap } from './server/src/routes/proxy.ts';

for (const [key, { modelDbId, lastUsed }] of stickySessionMap) {
  console.log(`Key ${key} → Model ${modelDbId}, last used ${new Date(lastUsed)}`);
}

```

*Note: This map is not exported in production builds and should only be used in development environments.*

## Summary

- **Sticky sessions** in FreeLLMAPI use a deterministic key (either `X-Session-Id` header or SHA-1 hash of the first message) to identify conversations.
- The `stickySessionMap` in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) caches the model assignment for up to **30 minutes** with a maximum of **500 entries**.
- The `getStickyModel` and `setStickyModel` functions in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) handle TTL validation and cleanup, ensuring memory efficiency.
- The **preferred model** is tried first in the routing chain, falling back to normal selection only if the sticky entry expires or the model is unavailable.

## Frequently Asked Questions

### How does FreeLLMAPI generate session keys without explicit headers?

When the `X-Session-Id` header is absent, the `getSessionKey` function computes a SHA-1 hash of the first user message's content. This hash becomes the session key, allowing conversations to remain sticky even when clients cannot track session IDs themselves, provided the initial message content remains consistent.

### What happens when the sticky session map reaches 500 entries?

When the map exceeds 500 entries, `setStickyModel` performs housekeeping by scanning and removing entries older than the 30-minute TTL. This bounded size prevents memory leaks in high-throughput environments while maintaining recent session affinity.

### How long do sticky sessions persist?

Sticky entries expire after **30 minutes of inactivity** (controlled by `STICKY_TTL_MS` in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts)). After expiration, the next request for that session generates a new sticky assignment, potentially routing to a different model.

### Can I disable sticky sessions for specific requests?

There is no explicit disable flag in the current implementation. To effectively bypass sticky sessions, omit the `X-Session-Id` header and vary the first user message content, which forces a new session key generation and prevents cache hits in the `stickySessionMap`.