Sticky Sessions Duration in FreeLLMAPI: How Long They Last and How They Work

FreeLLMAPI maintains sticky sessions for exactly 30 minutes, after which the conversation context expires and subsequent requests may be routed to a different model.

The tashfeenahmed/freellmapi repository implements an intelligent routing system that keeps multi-turn conversations bound to the same LLM provider for a specific duration. Understanding the duration of sticky sessions in FreeLLMAPI is essential for building stateful applications that require consistent model behavior across multiple API calls.

How Sticky Sessions Work in FreeLLMAPI

Sticky sessions ensure that when you start a conversation with a specific model, subsequent messages in the same session return to that same model rather than triggering a new routing decision. This prevents jarring mid-conversation provider switches that could disrupt context or change response characteristics.

The 30-Minute TTL Mechanism

The duration is hardcoded as a Time-To-Live (TTL) constant in the proxy route handler. In server/src/routes/proxy.ts, the system defines:

const STICKY_TTL_MS = 30 * 60 * 1000; // 30 min session TTL

This 30-minute window (1,800,000 milliseconds) represents the maximum idle time allowed before the session expires. When a request arrives, the server checks the stickySessionMap to see if the session ID has a recorded model entry. If the timestamp of the last use is newer than the STICKY_TTL_MS, the router reuses that model; otherwise, the entry expires and normal model selection resumes.

Session Tracking and Model Selection

The routing logic relies on two key files to maintain session state:

• server/src/routes/proxy.ts: Contains the stickySessionMap data structure that stores the model ID and timestamp for each active session
• server/src/services/router.ts: Implements getStickyModel and setStickyModel functions that check and update the sticky session bindings

According to the README documentation, this behavior ensures that "multi-turn conversations keep talking to the same model for 30 minutes before the context expires."

Implementation Details in the Source Code

The sticky session mechanism operates through a continuous cleanup cycle. While the stickySessionMap holds active bindings, a background process periodically purges entries older than 30 minutes to prevent memory leaks.

When processing a request, the router follows this logic:

1. Check for the X-Session-Id header
2. Look up the session ID in stickySessionMap
3. If found and timestamp is within STICKY_TTL_MS, route to the stored model
4. If expired or not found, execute normal model selection (fallback chain)
5. Update the timestamp for new or renewed sessions

The server/src/services/context-handoff.ts file references a related 3-hour hand-off TTL that works alongside the sticky session system, though the 30-minute sticky duration remains the primary binding mechanism for active conversations.

Practical Usage Example

To maintain a sticky session across multiple turns, clients must consistently send the same session identifier in the X-Session-Id header:

import fetch from 'node-fetch';

const response = await fetch('http://localhost:3001/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer <your-unified-api-key>',
    'Content-Type': 'application/json',
    'X-Session-Id': 'my-sticky-session-123',
  },
  body: JSON.stringify({
    model: 'auto',
    messages: [{role: 'user', content: 'Explain sticky sessions'}],
    stream: false,
  })
});

const data = await response.json();
console.log(data);

If subsequent requests using the same X-Session-Id arrive within 30 minutes, FreeLLMAPI automatically routes them to the original model selected in the first turn. After 30 minutes of inactivity, the next request triggers fresh model selection.

Summary

• Sticky sessions in FreeLLMAPI last exactly 30 minutes (1,800,000 ms) as defined by STICKY_TTL_MS in server/src/routes/proxy.ts
• Sessions are tracked in stickySessionMap with timestamps checked against the TTL on each request
• The router in server/src/services/router.ts uses getStickyModel to retrieve bindings and setStickyModel to update them
• Consistent use of the X-Session-Id header maintains session continuity across API calls
• After expiration, the cleanup loop removes entries and activates standard model selection logic

Frequently Asked Questions

What happens after 30 minutes of inactivity?

After 30 minutes without a request bearing the same session ID, the sticky session entry is purged from stickySessionMap. The next request with that session ID undergoes fresh model selection, potentially routing to a different provider based on availability, cost, or performance criteria.

How do I maintain a sticky session across multiple requests?

Include the identical X-Session-Id header value in every request of the conversation sequence. Ensure requests occur within the 30-minute window to prevent TTL expiration. The server automatically extends the session lifetime with each valid request.

Where is the sticky session data stored?

Session mappings reside in an in-memory stickySessionMap object within server/src/routes/proxy.ts. This transient storage is not persisted to disk; sessions survive only as long as the server process runs and the entries remain within the 30-minute TTL window.

Can I configure the sticky session duration?

No, the 30-minute duration is hardcoded as a constant (STICKY_TTL_MS) in the current implementation. Modifying the duration requires changing the source code in server/src/routes/proxy.ts and redeploying the server.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}