How FreeLLMAPI Handles Sticky Sessions for Conversations

FreeLLMAPI maintains conversation continuity across stateless HTTP requests by generating a UUID session identifier stored in server-side memory and returning it via the x-llmapi-session-id header, which clients echo on subsequent calls to retrieve previous context.

FreeLLMAPI is an open-source API layer that enables multi-turn LLM conversations without requiring WebSocket connections or persistent client-state management. The implementation relies on a lightweight server-side retention mechanism that maps unique session identifiers to conversation history and model parameters. This architecture allows the API to remain technically stateless while providing clients with the illusion of continuous dialogue across independent HTTP requests.

Session Identifier Generation and Storage

UUID Creation in request-retention.ts

When a client initiates a conversation, the server generates a short-lived UUID in server/src/services/request-retention.ts. This session-id is returned to the client in the response header x-llmapi-session-id, establishing the sticky session bond.

In-Memory Session Store

The service maintains a Map<string, SessionData> that associates each session identifier with the last request payload, selected model, and auxiliary metadata including token usage. This server-side storage eliminates the need for clients to manage conversation history locally.

Memory Management with pruneRequestAnalytics()

To prevent memory bloat, the system periodically invokes pruneRequestAnalytics() to remove entries older than the configurable TTL (default 15 minutes). This automatic cleanup ensures that abandoned sessions do not consume server resources indefinitely.

Middleware Integration and Context Restoration

Header Processing in router.ts

The router.ts file registers a pre-handler middleware that extracts the x-llmapi-session-id header from incoming requests. If the header exists and matches an entry in the retention map, the system retrieves the stored SessionData and prepares it for context merging.

Automatic Context Merging

Before forwarding requests to the LLM provider, the middleware merges the stored conversation history from the previous SessionData into the current request body. This effectively restores the previous context, allowing the model to reference earlier parts of the conversation without requiring the client to resend the full message history.

Security Isolation and Access Control

API Key Validation in auth.ts

The auth.ts service validates that each session belongs to the specific API key making the request. This API-key scoping prevents unauthorized cross-tenant access, ensuring that one client cannot hijack another user's conversation by guessing or stealing a session identifier.

Client Implementation Examples

Initiating a New Conversation

When making the first request, the client stores the returned session ID for subsequent calls:

// Client-side initiation
await fetch('https://api.free-llmapi.com/v1/completions', {
  method: 'POST',
  headers: { 
    'Content-Type': 'application/json', 
    'Authorization': `Bearer ${API_KEY}` 
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Explain sticky sessions' }]
  })
})
.then(res => {
  const sessionId = res.headers.get('x-llmapi-session-id');
  // Persist for continuity
  localStorage.setItem('llmapi-session-id', sessionId!);
  return res.json();
});

Continuing an Existing Conversation

Subsequent requests must include the x-llmapi-session-id header to maintain context:

const sessionId = localStorage.getItem('llmapi-session-id');

await fetch('https://api.free-llmapi.com/v1/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${API_KEY}`,
    'x-llmapi-session-id': sessionId!  // Sticky session header
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'How does it work?' }]
  })
})
.then(res => res.json());

Server-Side Middleware Flow

The simplified pre-handler logic in router.ts demonstrates how the system processes incoming session headers:

// router.ts pre-handler implementation
app.use((req, res, next) => {
  const sessionId = req.headers['x-llmapi-session-id'] as string;
  if (sessionId) {
    const stored = requestRetention.get(sessionId);
    if (stored) {
      // Merge previous messages with current request
      req.body.messages = [...stored.messages, ...req.body.messages];
    }
  }
  next();
});

Summary

• Session identifiers are generated as UUIDs in server/src/services/request-retention.ts and returned via the x-llmapi-session-id header.
• In-memory storage uses a Map<string, SessionData> to retain conversation context, token usage, and model selection.
• Automatic cleanup occurs through pruneRequestAnalytics(), which removes stale sessions after the default 15-minute TTL.
• Middleware integration in router.ts extracts session headers and merges stored context into active requests.
• Security isolation via auth.ts ensures sessions are scoped to specific API keys, preventing cross-tenant access.

Frequently Asked Questions

What happens if the client does not send the session header?

If the x-llmapi-session-id header is missing or invalid, FreeLLMAPI treats the request as a new conversation. The server generates a fresh session ID and returns it in the response headers, allowing the client to adopt this new session for subsequent turns if desired.

How long does FreeLLMAPI retain session data?

By default, FreeLLMAPI retains session data for 15 minutes before automatic expiration. The pruneRequestAnalytics() function in request-retention.ts periodically scans the in-memory store and removes entries exceeding this TTL, though administrators can configure this threshold.

Is conversation history shared between different API keys?

No. The auth.ts service validates that each session belongs exclusively to the API key that created it. This isolation prevents clients from accessing or hijacking conversations associated with other accounts, even if they possess the session identifier.

What data is stored in the SessionData object?

The SessionData structure stores the last request payload (including the full message history), the selected model identifier, and auxiliary metadata such as token usage statistics. This comprehensive storage enables true continuity without requiring the client to resend previous context.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}