What Are Sticky Sessions in FreeLLMAPI and How Do They Improve Conversation Consistency?

FreeLLMAPI implements sticky sessions by generating a unique session ID on the first request, storing conversation context in an in-memory Map, and retrieving that history on subsequent calls via the x-llmapi-session-id header, enabling stateless clients to maintain continuous LLM dialogues.

The open-source repository tashfeenahmed/freellmapi solves the challenge of maintaining conversational state across discrete HTTP requests through a lightweight sticky session mechanism. Unlike traditional stateless REST APIs that require clients to resend full message histories, FreeLLMAPI retains conversation context server-side, allowing clients to reference previous exchanges using a simple identifier. This architecture ensures that multi-turn conversations remain consistent without requiring complex client-side state management or persistent connections like WebSockets.

Session Lifecycle and Storage Mechanism

FreeLLMAPI's sticky session implementation centers on three core components: the session generator, the retention store, and the middleware injector.

UUID Generation and Response Headers

When a client makes an initial request to an LLM endpoint, the server generates a short-lived UUID that serves as the session ID. This identifier is created in server/src/services/request-retention.ts and returned to the client in the response header x-llmapi-session-id. The client must store this identifier locally to maintain continuity across subsequent requests.

In-Memory Session Store

The retention service maintains a Map<string, SessionData> that maps each active session ID to its corresponding conversation data. Each entry stores the last request payload, the selected model configuration, and auxiliary metadata such as token usage counts. This in-memory storage enables rapid context retrieval without database latency, though it requires careful memory management for high-throughput deployments.

Middleware Context Injection in router.ts

The routing layer in server/src/services/router.ts registers a pre-handler middleware that intercepts incoming requests before they reach the LLM handler. This middleware extracts the x-llmapi-session-id header from the request. If the header exists and matches an entry in the retention map, the stored SessionData is merged into the current request body, effectively appending the historical conversation context to the new user message.

Security and Session Isolation

FreeLLMAPI implements strict security boundaries to prevent session hijacking. In server/src/services/auth.ts, each session is scoped to a specific API key during validation. This isolation ensures that a client authenticating with one API key cannot access or modify session data belonging to another key, even if they somehow obtain a valid session ID from a different tenant.

Memory Management and TTL Configuration

To prevent memory exhaustion, server/src/services/request-retention.ts implements an automatic cleanup mechanism. The pruneRequestAnalytics() function runs periodically to remove stale entries from the session Map. By default, sessions expire after 15 minutes of inactivity, though this TTL is configurable. This garbage collection ensures that abandoned conversations do not consume server resources indefinitely while allowing active users to maintain long-running dialogues.

Client Implementation Examples

Initiating a Conversation

When making the first request, the client receives the session header which must be preserved for subsequent calls:

// First request - capture the session ID
await fetch('https://api.free-llmapi.com/v1/completions', {
  method: 'POST',
  headers: { 
    'Content-Type': 'application/json', 
    'Authorization': `Bearer ${API_KEY}` 
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Explain sticky sessions' }]
  })
})
.then(res => {
  const sessionId = res.headers.get('x-llmapi-session-id');
  // Persist for subsequent requests
  localStorage.setItem('llmapi-session-id', sessionId!);
  return res.json();
});

Continuing an Existing Session

Subsequent requests must include the session header to retrieve the stored context:

const sessionId = localStorage.getItem('llmapi-session-id');

await fetch('https://api.free-llmapi.com/v1/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${API_KEY}`,
    'x-llmapi-session-id': sessionId!  // Sticky session header
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'How does it work?' }]
  })
});

Server-Side Flow

The server merges stored context automatically before forwarding to the LLM provider:

// server/src/services/router.ts - simplified pre-handler
app.use((req, res, next) => {
  const sessionId = req.headers['x-llmapi-session-id'] as string;
  if (sessionId) {
    const stored = requestRetention.get(sessionId);
    if (stored) {
      // Prepend historical messages to current request
      req.body.messages = [...stored.messages, ...req.body.messages];
    }
  }
  next();
});

Summary

• FreeLLMAPI implements sticky sessions through a combination of UUID generation, in-memory storage, and middleware injection.
• The server/src/services/request-retention.ts service manages the session lifecycle, including automatic cleanup via pruneRequestAnalytics().
• Clients receive a x-llmapi-session-id header on the first response and must return it on subsequent requests to maintain continuity.
• Sessions are isolated by API key in server/src/services/auth.ts, preventing cross-tenant data access.
• A default 15-minute TTL balances conversation continuity with memory efficiency, automatically expiring inactive sessions.

Frequently Asked Questions

What happens if I forget to include the session header?

If the x-llmapi-session-id header is missing or invalid, server/src/services/router.ts treats the request as a new conversation. The server generates a fresh session ID and responds without historical context, meaning the LLM will not have access to previous messages from earlier exchanges.

How long do sticky sessions last in FreeLLMAPI?

By default, sessions expire after 15 minutes of inactivity. The pruneRequestAnalytics() function in server/src/services/request-retention.ts periodically scans the session Map and removes entries older than the configured TTL. Active users can maintain long conversations indefinitely by ensuring their requests occur within this window.

Are sticky sessions shared between different API keys?

No. According to the validation logic in server/src/services/auth.ts, sessions are strictly scoped to the API key used during the initial request. Even if another client obtains a valid session ID, they cannot access that session's data without authenticating with the same API key, ensuring tenant isolation.

How does the system prevent memory leaks from abandoned sessions?

The retention service implements automatic garbage collection through pruneRequestAnalytics(), which runs on a scheduled interval to purge expired entries from the Map<string, SessionData>. This mechanism, combined with the configurable TTL, ensures that unused session data is reclaimed automatically, preventing memory bloat in long-running server instances.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}