# How FreeLLMAPI Handles Context Handoff When Switching Models Mid-Conversation

> Learn how FreeLLMAPI seamlessly handles context handoff when switching models mid-conversation. Discover how recent history is transferred and summarized for uninterrupted AI chats. Explore FreeLLMAPI now.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: internals
- Published: 2026-06-22

---

**FreeLLMAPI automatically transfers recent conversation history to new models during mid-conversation switches by injecting a system handoff message containing a summarized context window up to 6,000 characters.**

FreeLLMAPI's intelligent routing layer can transparently shift traffic between different LLM providers when rate limits hit or better models become available. To maintain conversational continuity during these transitions, the platform implements a **context handoff** mechanism that preserves up to 12 recent message turns and injects them as a system prompt when `FREELLMAPI_CONTEXT_HANDOFF` is set to `on_model_switch`. This process is fully implemented in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts).

## Enabling Context Handoff

The handoff system is controlled via environment configuration. By default, the feature is disabled to minimize token usage.

### Configuration Mode

The `getContextHandoffMode()` function reads the `FREELLMAPI_CONTEXT_HANDOFF` environment variable (lines 27-30) to determine behavior:

- **`off`**: Disables context handoff entirely
- **`on_model_switch`**: Activates automatic context injection when the routing layer selects a different model than the previous turn

```env
FREELLMAPI_CONTEXT_HANDOFF=on_model_switch

```

When enabled, the router reserves additional token headroom using the `HANDOFF_MAX_TOKENS` constant (lines 21-24), approximately 1,500 tokens, to accommodate the injected summary.

## The Context Handoff Workflow

The handoff process operates across multiple request cycles, tracking session state in an in-memory `Map` with a 3-hour TTL (`SESSION_TTL_MS`).

### Recording Incoming Messages

Before routing each request, `recordIncomingMessages()` (lines 56-78) persists the conversation history. It stores the last 12 user and assistant messages, trimming each to 500 characters maximum. This data is keyed by the session identifier and maintained in an in-memory store limited to 500 entries (`MAX_STORE_SIZE`).

```ts
import { recordIncomingMessages } from './services/context-handoff';

recordIncomingMessages(sessionKey, [
  { role: 'user', content: 'What is the weather in London?' },
  { role: 'assistant', content: 'It is 12°C and partly cloudy.' },
]);

```

### Tracking Model Selection

After a provider successfully returns a response, `recordSuccessfulModel()` (lines 59-71) stores the `lastModelKey` that served the request. This enables the system to detect when the next request routes to a different provider.

```ts
import { recordSuccessfulModel } from './services/context-handoff';

recordSuccessfulModel(sessionKey, 'gpt-4');

```

### Injecting Context on Model Switches

The `maybeInjectContextHandoff()` function (lines 111-156) implements the core injection logic. When invoked by the router ([`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts)), it performs the following validation:

1. Verifies the mode is `on_model_switch` and a `sessionKey` exists
2. Retrieves the stored `SessionContext` and compares `lastModelKey` against the newly selected model
3. Guards against duplicate handoff messages
4. Generates a summary limited to **6,000 characters** via `buildSummary()` (lines 94-100)
5. Inserts a system-role handoff message immediately after existing system prompts

```ts
import { maybeInjectContextHandoff, getContextHandoffMode } from './services/context-handoff';

const mode = getContextHandoffMode();
const { messages, injected, injectedTokens } = maybeInjectContextHandoff({
  mode,
  sessionKey,
  messages: incomingMessages,
  selectedModelKey: 'claude-2',
});

```

If `injected` returns `true`, the `messages` array now contains the handoff system message.

## The Handoff Message Format

When a switch is detected (e.g., from `gpt-4` to `claude-2`), FreeLLMAPI constructs a system message that instructs the new model to continue the conversation without restarting:

```

FreeLLMAPI context handoff:
You are taking over an ongoing conversation from another model (gpt-4 → claude-2).
Continue the user's task using the conversation context already provided in this request.
Do not restart the task, re-ask already answered setup questions, or discard prior tool results.
Respect the user's latest message as the highest-priority instruction.

Recent session summary:
User: How do I reset my password?
Assistant: You can reset it via the Settings page…
...

```

The summary is assembled by concatenating the stored trimmed messages and truncating to the `MAX_HANDOFF_CHARS` limit.

## Token Budget and Storage Constraints

The system implements strict limits to prevent context overflow:

- **Message Retention**: Maximum 12 recent turns per session
- **Character Limit**: 500 characters per message, 6,000 characters total for the handoff summary
- **Token Reservation**: `HANDOFF_MAX_TOKENS` constant (≈1,500 tokens) reserved by the router for handoff injection
- **Storage TTL**: Sessions expire after 3 hours (`SESSION_TTL_MS`)
- **Capacity**: In-memory store limited to 500 entries (`MAX_STORE_SIZE`)

The `hasPriorModel()` helper (lines 102-109) allows the router to check if a session has previous model history, enabling proactive token headroom padding before the injection step.

```ts
import { HANDOFF_MAX_TOKENS } from './services/context-handoff';

const tokenBudget = estimateTokens(messages) + (injected ? HANDOFF_MAX_TOKENS : 0);

```

## Summary

- **Context handoff** maintains conversation continuity when FreeLLMAPI switches models mid-conversation by injecting a system summary message
- The feature is controlled via `FREELLMAPI_CONTEXT_HANDOFF` environment variable with modes `off` or `on_model_switch`
- Up to 12 recent messages are stored in-memory with a 3-hour TTL, trimmed to 500 characters each
- The `maybeInjectContextHandoff()` function in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts) handles detection and injection when `lastModelKey` differs from the current selection
- Handoff summaries are capped at 6,000 characters (≈1,500 tokens) and inserted after existing system prompts

## Frequently Asked Questions

### What triggers a context handoff in FreeLLMAPI?

A context handoff triggers when the routing layer selects a different model than the one that handled the previous request in the same session, provided `FREELLMAPI_CONTEXT_HANDOFF` is set to `on_model_switch`. The system compares the stored `lastModelKey` against the newly selected model key to detect the transition.

### How much conversation history does FreeLLMAPI preserve during a handoff?

FreeLLMAPI preserves up to 12 recent user and assistant messages per session. Each message is trimmed to 500 characters maximum, and the final handoff summary is limited to 6,000 characters (approximately 1,500 tokens). This balances context continuity with token economy.

### Is context handoff enabled by default?

No, context handoff is disabled by default. You must explicitly set the environment variable `FREELLMAPI_CONTEXT_HANDOFF=on_model_switch` to enable the feature. When disabled, model switches occur without injecting historical context, and the new model sees only the current request messages.

### How does FreeLLMAPI prevent duplicate handoff messages?

The `maybeInjectContextHandoff()` function includes deduplication logic that guards against injecting multiple handoff messages for the same conversation turn. It checks the existing message array to ensure a handoff system message has not already been inserted before adding the context summary.