How Sticky Sessions Maintain Conversation Continuity in FreeLLMAPI

FreeLLMAPI maintains conversation continuity across stateless HTTP requests by generating a unique session identifier stored server-side in an in-memory Map, which subsequent requests retrieve via the x-llmapi-session-id header to restore previous context.

FreeLLMAPI (tashfeenahmed/freellmapi) implements a lightweight sticky-session layer that enables multi-turn LLM conversations without requiring persistent connections. By leveraging an in-memory retention service and custom middleware, the API associates sequential requests with a single conversation thread using a transient session identifier that clients persist between calls.

Sticky Session Architecture

Session Identifier Generation

When a client initiates a new conversation, the server generates a short-lived UUID that serves as the session-id. This logic resides in server/src/services/request-retention.ts, which creates the identifier and returns it to the client in the x-llmapi-session-id response header. The generation occurs only on the first request of a conversation chain, establishing the sticky session anchor point.

In-Memory Session Storage

The retention service maintains a Map<string, SessionData> that maps session identifiers to conversation state. Each entry stores the last request payload, the selected model configuration, and auxiliary metadata such as token usage statistics. This server-side storage strategy keeps the API itself stateless while centralizing conversation history in a single process-local data structure.

Middleware Context Restoration

The router defined in server/src/services/router.ts registers a pre-handler middleware that intercepts incoming requests and extracts the x-llmapi-session-id header. If the header exists and matches an entry in the retention map, the stored SessionData merges into the current request body. This injection restores the previous conversation context, allowing the LLM to reference earlier messages as if the connection had remained open.

Implementation Examples

Initiating a Conversation (First Request)

Clients receive the session identifier in the response headers and must store it for subsequent calls:

// Client-side initialization
await fetch('https://api.free-llmapi.com/v1/completions', {
  method: 'POST',
  headers: { 
    'Content-Type': 'application/json', 
    'Authorization': `Bearer ${API_KEY}` 
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Explain sticky sessions' }]
  })
})
.then(res => {
  const sessionId = res.headers.get('x-llmapi-session-id');
  // Persist for conversation continuity
  localStorage.setItem('llmapi-session-id', sessionId!);
  return res.json();
});

Continuing a Conversation (Subsequent Requests)

To maintain continuity, clients must include the stored session identifier in the request headers:

const sessionId = localStorage.getItem('llmapi-session-id');

await fetch('https://api.free-llmapi.com/v1/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${API_KEY}`,
    'x-llmapi-session-id': sessionId!  // Sticky session header
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [
      { role: 'assistant', content: 'Sticky sessions allow...' },
      { role: 'user', content: 'How does the server store this?' }
    ]
  })
});

Server-Side Context Merging

The middleware in server/src/services/router.ts handles the restoration logic:

// Simplified pre-handler flow
app.use((req, res, next) => {
  const sessionId = req.headers['x-llmapi-session-id'] as string;
  if (sessionId) {
    const stored = requestRetention.get(sessionId);
    if (stored) {
      // Prepend previous messages to current request
      req.body.messages = [...stored.messages, ...req.body.messages];
    }
  }
  next();
});

Security and Lifecycle Management

API Key Isolation

The server/src/services/auth.ts module validates API keys before processing session retrieval, ensuring that session data remains scoped to the originating client. This prevents cross-tenant session hijacking by verifying that the stored SessionData belongs to the authenticated API key holder.

Automatic Cleanup

To prevent memory bloat, server/src/services/request-retention.ts implements pruneRequestAnalytics(), a background job that removes entries older than the configurable TTL (default 15 minutes). This automatic expiration ensures that abandoned conversations do not exhaust server resources while active sessions remain available for multi-turn interactions.

Summary

• Session Generation: server/src/services/request-retention.ts creates UUID session identifiers returned via the x-llmapi-session-id header.
• State Storage: An in-memory Map<string, SessionData> holds conversation context server-side, enabling stateless HTTP operations.
• Context Restoration: The router middleware in server/src/services/router.ts merges stored history into new requests when the session header is present.
• Security Boundary: Sessions are isolated by API key in server/src/services/auth.ts, preventing unauthorized access to conversation data.
• Resource Management: Automatic pruning removes stale sessions after 15 minutes, balancing continuity with memory efficiency.

Frequently Asked Questions

What is the default session TTL in FreeLLMAPI?

FreeLLMAPI retains sticky session data for 15 minutes by default. The pruneRequestAnalytics() function in server/src/services/request-retention.ts periodically scans the session Map and deletes entries exceeding this TTL, ensuring that inactive conversations expire automatically while active users experience uninterrupted continuity.

How does FreeLLMAPI prevent session hijacking?

The system isolates sessions by API key through validation logic in server/src/services/auth.ts. Even if a malicious actor obtains a valid session-id, the middleware verifies that the requesting API key matches the key associated with the stored SessionData before injecting context, effectively sandboxing conversations to authorized clients only.

Can sticky sessions work across multiple server instances?

The current implementation uses an in-memory Map within a single process, meaning sticky sessions do not automatically synchronize across horizontally scaled instances. For distributed deployments, the architecture would require replacing the local Map with a shared external store (such as Redis) while maintaining the same x-llmapi-session-id header protocol.

Which HTTP header carries the session identifier?

FreeLLMAPI uses the x-llmapi-session-id header for session tracking. The server returns this header in the response to the initial request, and clients must include it in subsequent requests to the same endpoint to maintain conversation continuity through the sticky session mechanism.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}