# How FreeLLMAPI Context Handoff Preserves Conversations When Switching Models

> Discover how FreeLLMAPI context handoff ensures seamless conversation continuity when switching models. Learn how session summaries maintain your chat history.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: internals
- Published: 2026-06-30

---

**FreeLLMAPI preserves conversation continuity across model switches by injecting a compact session summary as a system message whenever the selected model differs from the one that handled the previous turn.**

FreeLLMAPI, an open-source LLM proxy from `tashfeenahmed/freellmapi`, solves a common multi-model routing problem: when a user switches from one model to another mid-conversation, the new model lacks the implicit context of prior turns. The **context handoff** feature bridges this gap without exposing provider-specific state management to the client.

## Enabling the Context Handoff Feature

The entire flow is gated by the `FREELLMAPI_CONTEXT_HANDOFF` environment variable. When set to `on_model_switch`, FreeLLMAPI evaluates every request for a potential handoff. Any other value, or an unset variable, completely disables the logic.

According to the source in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts), the function `getContextHandoffMode` reads this variable and returns the active mode (lines 27-30). The proxy only proceeds with injection if this mode matches the required trigger.

```bash

# .env or exported before starting the server

export FREELLMAPI_CONTEXT_HANDOFF=on_model_switch

```

## How the Context Handoff Mechanism Works

The core implementation lives in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts). It operates as a six-step pipeline that runs per request.

### 1. Tracking Recent Messages Per Session

Every inbound request records the last ~12 messages in an in-memory **`Map`** keyed by the client-provided **`sessionKey`**. The helper **`recordIncomingMessages`** trims each message to roughly 500 characters before storage to bound memory growth (lines 56-79).

Clients must supply a stable **`sessionKey`** (for example, a UUID) across turns. Without this key, the service cannot correlate turns and the handoff window is lost.

### 2. Remembering the Last Successful Model

After a model finishes a turn successfully, **`recordSuccessfulModel`** stores its identifier as a composite **`lastModelKey`** string—formatted as `platform:modelId`—inside the same session entry (lines 59-73). This stored key becomes the baseline for comparison on the next request.

### 3. Detecting a Model Switch

On each subsequent request, **`maybeInjectContextHandoff`** compares the stored **`lastModelKey`** against the newly chosen **`selectedModelKey`** (lines 111-154). If the keys differ, the service flags a handoff event and proceeds to summary generation.

### 4. Building a Session Summary

If a switch is detected, **`buildSummary`** concatenates the stored, trimmed messages into a single string capped at 6000 characters (lines 94-100). This truncated history is deliberately compact to avoid consuming excessive context-window space.

### 5. Injecting the System-Message Handoff

The summary is wrapped in a system-role message prefixed with `FreeLLMAPI context handoff:`. The proxy inserts this message *after* any leading system prompts already present in the request, preserving the provider’s native system prompt ordering (lines 134-152).

The injected message instructs the new model to continue the user's task without restarting, re-asking setup questions, or discarding prior tool results.

### 6. Accounting for Token Headroom

To prevent downstream context-window overflows, the proxy inflates the routing token estimate by **`HANDOFF_MAX_TOKENS`** (approximately 1600 tokens) whenever a handoff could be injected. This padding is applied in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) (lines 54-60) before the request reaches the target provider.

## Practical Example: Switching from GPT-4o to Claude

The following pattern demonstrates a conversation that begins on OpenAI and continues on Anthropic.

First, the client initiates a turn using a specific `sessionKey`:

```ts
import axios from 'axios';

// The `sessionKey` is a stable identifier for a user conversation (e.g. a UUID).
const payload = {
  model: 'gpt-4o',          // first turn – model A
  messages: [{ role: 'user', content: 'Explain quantum tunnelling.' }],
  sessionKey: '123e4567-e89b-12d3-a456-426614174000',
};

await axios.post('https://api.freellmapi.com/v1/chat/completions', payload);

```

On the next turn, the client requests a different model while keeping the same `sessionKey`:

```ts
const secondPayload = {
  model: 'anthropic/claude-3',  // model B – different platform
  messages: [
    { role: 'assistant', content: '... (previous answer)' },
    { role: 'user', content: 'Now apply that to a semiconductor device.' },
  ],
  sessionKey: '123e4567-e89b-12d3-a456-426614174000',
};

await axios.post('https://api.freellmapi.com/v1/chat/completions', secondPayload);

```

Because `lastModelKey` (`gpt-4o`) differs from the selected model (`anthropic/claude-3`), FreeLLMAPI injects a system message similar to:

```

FreeLLMAPI context handoff:
You are taking over an ongoing conversation from another model (gpt-4o → anthropic/claude-3).
Continue the user's task using the conversation context already provided in this request.
Do not restart the task, re‑ask already answered setup questions, or discard prior tool results.
Respect the user's latest message as the highest‑priority instruction.

Recent session summary:
User: Explain quantum tunnelling.
Assistant: ... (truncated)
User: Now apply that to a semiconductor device.

```

The downstream Claude model receives this preamble first, allowing it to continue seamlessly.

## Key Source Files and Responsibilities

Several modules collaborate to make the context handoff feature reliable:

- **[`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts)** — Contains the core logic, including mode handling (`getContextHandoffMode`), message trimming (`recordIncomingMessages`), success tracking (`recordSuccessfulModel`), summary generation (`buildSummary`), and injection (`maybeInjectContextHandoff`).
- **[`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts)** — Orchestrates the optional injection and applies the `HANDOFF_MAX_TOKENS` padding to token estimates (lines 54-60).
- **[`server/src/services/request-retention.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/request-retention.ts)** — Provides the `sessionKey` plumbing that `recordIncomingMessages` depends on to key the in-memory store.
- **[`server/src/services/model-listing.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/model-listing.ts)** — Supplies the model-selection pipeline that yields the `selectedModelKey` used for switch detection.

## Summary

- **FreeLLMAPI context handoff** preserves multi-turn conversations when routing switches to a different model mid-session.
- The feature requires `FREELLMAPI_CONTEXT_HANDOFF=on_model_switch` and a stable client-provided **`sessionKey`**.
- **`recordIncomingMessages`** tracks the last ~12 truncated messages in-memory per session, while **`recordSuccessfulModel`** remembers the prior model identifier.
- **`maybeInjectContextHandoff`** detects model switches and injects a compact system summary after existing system prompts.
- The proxy reserves **`HANDOFF_MAX_TOKENS`** (~1600 tokens) in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) to protect downstream context-window limits.

## Frequently Asked Questions

### What triggers a context handoff in FreeLLMAPI?

A context handoff triggers when two conditions are met: the environment variable `FREELLMAPI_CONTEXT_HANDOFF` is set to `on_model_switch`, and the `lastModelKey` stored for the current `sessionKey` differs from the newly selected model's `platform:modelId` composite key. The check happens inside `maybeInjectContextHandoff` in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts) (lines 111-154).

### How does the context handoff feature handle token limits?

The service caps the generated session summary at 6000 characters during `buildSummary` and preemptively inflates the routing token estimate by `HANDOFF_MAX_TOKENS` (approximately 1600 tokens) in [`server/src/routes/proxy.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/proxy.ts) (lines 54-60). This padding ensures the downstream provider's context-window validation accounts for the extra system message before the request is forwarded.

### Can I disable context handoff entirely?

You can disable the feature by unsetting the `FREELLMAPI_CONTEXT_HANDOFF` environment variable or by setting it to any value other than `on_model_switch`. The helper `getContextHandoffMode` in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts) (lines 27-30) evaluates this variable, and the proxy skips all injection logic when the mode does not match.

### Why is the `sessionKey` parameter required for handoff?

The in-memory store is a `Map` keyed by `sessionKey`. `recordIncomingMessages` uses this key to correlate turns and accumulate conversation history (lines 56-79). Without a stable `sessionKey` across requests, FreeLLMAPI cannot retrieve the prior model identifier or message window, so no handoff summary can be generated.