# How Sticky Sessions Maintain Conversation Continuity in FreeLLMAPI

> Learn how FreeLLMAPI uses sticky sessions and a unique session ID header to ensure seamless conversation continuity between stateless HTTP requests.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: deep-dive
- Published: 2026-06-24

---

**FreeLLMAPI maintains conversation continuity across stateless HTTP requests by generating a unique session identifier stored in an in-memory Map, which the router middleware retrieves via the `x-llmapi-session-id` header to merge previous context with new requests.**

FreeLLMAPI is an open-source API gateway that enables multi-turn LLM conversations without persistent connections. The repository implements a lightweight **sticky session** mechanism that stores conversation state server-side while keeping the API stateless. This approach allows clients to maintain dialogue continuity by including a session identifier in subsequent requests.

## Session Identification and Header Generation

When a client initiates a conversation, the server generates a short-lived UUID that serves as the **session identifier**. This logic resides in [`server/src/services/request-retention.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/request-retention.ts), where the system creates a new session and returns it in the response header `x-llmapi-session-id`.

The client must store this identifier and include it in subsequent requests to maintain continuity. The generation occurs automatically on the first request to any LLM endpoint, requiring no additional configuration from the client beyond standard authentication.

## In-Memory Session Storage Architecture

The **request-retention service** maintains conversation state using a `Map<string, SessionData>` structure that maps session identifiers to their associated data. Each entry stores the last request payload, the selected model, and auxiliary metadata such as token usage.

This in-memory storage provides fast lookup times for middleware integration but requires careful memory management. The system scopes all session data to a specific API key, ensuring complete isolation between different clients and preventing unauthorized access to conversation history.

## Middleware Integration and Context Restoration

The routing layer in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) registers a pre-handler middleware that processes incoming session headers. When a request contains the `x-llmapi-session-id` header, the middleware extracts the identifier and queries the retention Map.

If the session exists, the stored **SessionData** merges into the current request, effectively appending the previous conversation context to the new message array. This restoration happens transparently before the request reaches the LLM provider, allowing the model to receive the full conversation history despite individual HTTP requests being stateless.

## Security Boundaries and Automatic Cleanup

Session security relies on API key validation implemented in [`server/src/services/auth.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/auth.ts). Each session associates strictly with the API key used during its creation, preventing session hijacking across different client accounts.

To prevent memory bloat, [`request-retention.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/request-retention.ts) implements a background cleanup mechanism through `pruneRequestAnalytics()`. This function periodically removes entries exceeding a configurable TTL, which defaults to 15 minutes. The automatic expiration ensures that abandoned sessions do not consume server resources indefinitely while maintaining active conversations for a reasonable interaction window.

## Implementation Examples

### Initiating a Conversation

When making the first request, extract the session identifier from the response headers:

```typescript
const response = await fetch('https://api.free-llmapi.com/v1/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${API_KEY}`
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Explain sticky sessions' }]
  })
});

const sessionId = response.headers.get('x-llmapi-session-id');
localStorage.setItem('llmapi-session-id', sessionId!);
const data = await response.json();

```

### Continuing an Existing Session

Include the stored session identifier in subsequent requests to maintain context:

```typescript
const sessionId = localStorage.getItem('llmapi-session-id');

const response = await fetch('https://api.free-llmapi.com/v1/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${API_KEY}`,
    'x-llmapi-session-id': sessionId!
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'How does that work?' }]
  })
});

const data = await response.json();

```

### Server-Side Middleware Flow

The router middleware handles session restoration before processing the LLM request:

```typescript
// From server/src/services/router.ts
app.use((req, res, next) => {
  const sessionId = req.headers['x-llmapi-session-id'] as string;
  
  if (sessionId) {
    const stored = requestRetention.get(sessionId);
    if (stored) {
      // Merge previous messages with current request
      req.body.messages = [...stored.messages, ...req.body.messages];
    }
  }
  
  next();
});

```

### Key Source Files

- [`server/src/services/request-retention.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/request-retention.ts): Generates session IDs, manages the in-memory Map, and handles pruning via `pruneRequestAnalytics()`
- [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts): Implements middleware that extracts headers and merges SessionData into requests
- [`server/src/services/auth.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/auth.ts): Validates API keys and enforces session isolation boundaries

## Summary

- **Sticky sessions** in FreeLLMAPI use UUID-based identifiers passed via the `x-llmapi-session-id` header to track conversations across HTTP requests.
- The **in-memory Map** stores conversation context in [`server/src/services/request-retention.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/request-retention.ts), enabling fast retrieval and context restoration.
- **Router middleware** automatically merges historical messages with new requests when valid session headers are present.
- **Security isolation** binds sessions to specific API keys, while `pruneRequestAnalytics()` enforces a default 15-minute TTL to prevent memory leaks.

## Frequently Asked Questions

### How long do sticky sessions last in FreeLLMAPI?

Sticky sessions expire automatically after a configurable time-to-live (TTL) that defaults to 15 minutes. The `pruneRequestAnalytics()` function in [`server/src/services/request-retention.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/request-retention.ts) periodically removes expired entries from the in-memory store. Clients must initiate new conversations if their session expires between requests.

### What happens if I don't include the session header in subsequent requests?

If the `x-llmapi-session-id` header is missing, the router middleware treats the request as a new conversation. The system will generate a fresh session identifier and return it in the response headers, but the LLM will not receive the previous conversation context, resulting in a disconnected dialogue.

### Can multiple clients access the same sticky session?

No. The **auth service** in [`server/src/services/auth.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/auth.ts) strictly associates each session with the API key used during its creation. Even if another client obtains the session identifier, the middleware validates the API key against the stored session data before merging context, preventing cross-client session hijacking.

### Is the session data persisted to disk or database?

No. FreeLLMAPI implements **in-memory storage only** using a JavaScript Map object. This design prioritizes speed and simplicity but means session data is lost if the server restarts. The system is optimized for short-term conversational continuity rather than long-term persistence.