# What Is FREELLMAPI_CONTEXT_HANDOFF? Context Preservation Across Model Switches in FreeLLM

> Discover FREELLMAPI_CONTEXT_HANDOFF, a feature preserving conversation context between LLM model switches. Learn how it simplifies multi-model interactions.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: feature-explanation
- Published: 2026-06-27

---

**The `FREELLMAPI_CONTEXT_HANDOFF` environment variable enables context hand-off, a mechanism that preserves conversation history when routing requests between different LLM models by automatically injecting a system message containing a summary of recent turns.**

In multi-model LLM deployments, requests often bounce between different providers or model versions based on availability, cost, or capability. Without intervention, each new model starts with zero context, forcing users to repeat information. The `FREELLMAPI_CONTEXT_HANDOFF` feature in the `tashfeenahmed/freellmapi` repository solves this by maintaining an in-memory record of recent conversations and seamlessly briefing new models when a switch occurs.

## How Context Hand-Off Works

When `FREELLMAPI_CONTEXT_HANDOFF` is set to `on_model_switch`, the server activates a three-stage pipeline that runs during request processing.

### Tracking Conversation History

The system records every incoming request to build a short-term memory buffer. The `recordIncomingMessages` function in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts) stores recent user and assistant turns in an in-memory dictionary keyed by the session identifier. To prevent unbounded memory growth, the history is trimmed to a safe size before storage.

```typescript
// Located at server/src/services/context-handoff.ts (lines 56-78)
recordIncomingMessages(sessionKey, messages);

```

Simultaneously, after a provider returns a successful response, `recordSuccessfulModel` captures the model key that generated the reply. This creates a breadcrumb trail showing which model last handled the session.

```typescript
// Located at server/src/services/context-handoff.ts (lines 59-71)
recordSuccessfulModel({
  sessionKey: req.headers['x-session-id'] ?? '',
  modelKey: chosenModel.key,
});

```

### Detecting Model Switches

Before forwarding a request to a provider, the routing layer in [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts) invokes `maybeInjectContextHandoff`. This helper compares the `selectedModelKey` against the stored `lastModelKey` for that session. If they differ, the logic determines that a hand-off is required.

### Injecting Hand-Off Messages

When a switch is detected, `maybeInjectContextHandoff` (lines 111–156 in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts)) prepends a system message to the conversation array. This message contains:
- A brief description announcing the hand-off
- A summary of the recent conversation (up to approximately 6,000 characters)
- Explicit instructions directing the new model to continue the ongoing task

The modified message array is then forwarded to the new model, allowing it to respond as if it had been part of the conversation from the start.

## Configuration and Activation

Context hand-off is opt-in via environment variable. By default, `getContextHandoffMode` returns `'off'`, disabling all tracking and injection logic.

```bash

# Enable context hand-off on model switches

FREELLMAPI_CONTEXT_HANDOFF=on_model_switch

```

To check the current mode programmatically:

```typescript
import { getContextHandoffMode } from './services/context-handoff.js';

const mode = getContextHandoffMode(); // Returns 'off' or 'on_model_switch'

```

## Token Budget Safety and Constraints

Hand-off messages consume tokens, so the implementation enforces strict ceilings to prevent quota violations. The constant `MAX_HANDOFF_CHARS` (set to 6,000) bounds the summary length, while `HANDOFF_MAX_TOKENS` (exported at lines 20–24 of [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts)) provides a pre-computed token ceiling. The routing layer uses this value to reserve tokens during quota checks before invoking the provider.

## Practical Implementation Examples

### Basic Router Integration

The following pattern shows how the proxy layer integrates the hand-off mechanism during request processing:

```typescript
import { getContextHandoffMode, maybeInjectContextHandoff } from './services/context-handoff.js';

// Inside the request-proxy logic
const mode = getContextHandoffMode();
const { messages, injected } = maybeInjectContextHandoff({
  mode,
  sessionKey: req.headers['x-session-id'] ?? '',
  messages: req.body.messages,          // Original user/assistant messages
  selectedModelKey: chosenModel.key,    // The model we are about to call
});

if (injected) {
  console.log('Context hand-off injected for model switch');
}
// Forward to provider with potentially modified messages
forwardRequestToProvider({ ...req.body, messages });

```

### Recording Successful Completion

After receiving a successful response from a provider, notify the hand-off service so it can update the last-known model for the session:

```typescript
import { recordSuccessfulModel } from './services/context-handoff.js';

recordSuccessfulModel({
  sessionKey: req.headers['x-session-id'] ?? '',
  modelKey: chosenModel.key,
});

```

### Testing the Hand-Off Logic

Use the exported test helpers to verify behavior without persisting state between test runs:

```typescript
import { 
  _clearStoreForTesting, 
  maybeInjectContextHandoff, 
  getContextHandoffMode,
  recordIncomingMessages,
  recordSuccessfulModel 
} from './services/context-handoff.js';

// Reset in-memory store
_clearStoreForTesting();

// Simulate first request handled by modelA
recordIncomingMessages('sess1', [{role:'user', content:'Hello'}]);
recordSuccessfulModel({sessionKey:'sess1', modelKey:'modelA'});

// Simulate switch to modelB
const result = maybeInjectContextHandoff({
  mode: getContextHandoffMode(),
  sessionKey: 'sess1',
  messages: [{role:'user', content:'Continue'}],
  selectedModelKey: 'modelB',
});

console.assert(result.injected === true, 'Hand-off should inject on model switch');

```

## Summary

- **Context Preservation**: `FREELLMAPI_CONTEXT_HANDOFF` maintains conversation continuity when FreeLLMAPI routes requests to different models.
- **Activation**: Set the environment variable to `on_model_switch` to enable tracking and injection.
- **Core Files**: Implementation lives in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts), integrated by [`server/src/services/router.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/router.ts).
- **Safety**: Built-in limits (`MAX_HANDOFF_CHARS` = 6,000) and token exports prevent quota overruns.
- **Storage**: In-memory session store keyed by session identifier tracks recent messages and the last successful model.

## Frequently Asked Questions

### What happens if FREELLMAPI_CONTEXT_HANDOFF is not set?

If the variable is undefined or set to any value other than `on_model_switch`, `getContextHandoffMode` returns `'off'`. The system skips all tracking and injection logic, and each model request receives only the messages explicitly provided in the API call.

### Does context hand-off work across server restarts?

No. The implementation uses an in-memory store (`_clearStoreForTesting` confirms this is a runtime memory structure). When the Node.js process restarts, the conversation history and last-model mappings are lost. For persistent context, you would need to implement external storage.

### How much does the hand-off message increase token usage?

The injected system message is capped at approximately 6,000 characters (`MAX_HANDOFF_CHARS`), and the service exports `HANDOFF_MAX_TOKENS` to help the routing layer reserve budget. Typically, the overhead is roughly 1,500 tokens or fewer, depending on conversation density.

### Can I customize the hand-off message content?

Currently, the hand-off message template is hardcoded within `maybeInjectContextHandoff` in [`server/src/services/context-handoff.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/context-handoff.ts). To customize the prompt instructions or summary format, you would need to modify the source code in that file, specifically the logic around line 111 where the system message is constructed.